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Abstract 


This  project  studied  the  incorporation  of  military  occupational  data  into  generic  human  performance 
modeling  software,  the  Integrated  Performance  Modelling  Environment  (IPME).  It  has  explored  the  use 
of  modeling  and  simulation  (M&S)  for  addressing  the  Canadian  Forces  (CF)  personnel  and  manpower 
issues.  A  set  of  Canadian  Air  Force  occupational  specification  data  were  integrated  into  IPME.  This 
report  documents  an  effort  to  validate  this  new  modeling  capability.  An  IPME  model  of  an  Unmanned 
Aerial  Vehicle  (UAV)  mission  was  created.  The  modeled  crew  consisted  of  operators  defined  by  job- 
related  task,  skill  and  knowledge  statements  obtained  from  the  occupational  data.  For  this  IPME  model, 
we  used  a  job  similarity  index  to  predict  operator  performance.  To  confirm  the  validity  of  this  approach, 
we  planned  to  replicate  the  same  UAV  model  in  the  US  Army’s  IMPRINT  and  compare  the  performance 
predictions  made  by  the  two  modeling  toolkits.  However,  due  to  the  lack  of  access  to  IMPRINT,  the 
Integrated  Simulation  Manpower  Analysis  Tool  (ISMAT)  was  used  instead.  As  a  personnel  modeling 
tool,  ISMAT  was  developed  primarily  for  targeting  naval  applications.  This  limited  our  ability  to  validate 
the  IPME  UAV  model  which  used  Air  Force  occupational  data.  As  a  result,  the  current  study  focused  on 
comparing  the  personnel  modeling  approaches  adopted  by  IPME  and  ISMAT.  The  differences  are 
documented,  and  they  provide  insight  into  future  IPME  research  and  development. 
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Resume 


L’objet  du  projet  etait  d’etudier  l’integration  de  donnees  sur  les  groupes  professionnels  militaires  dans  un 
logiciel  de  modelisation  generique  de  la  performance  humaine  appele  Integrated  Performance  Modelling 
Environment  (IPME).  Ce  projet  a  permis  d’explorer  l’utilisation  de  la  modelisation  et  de  la  simulation 
(M&S)  pour  tenter  de  resoudre  les  problemes  de  personnel  et  de  main-d’ceuvre  au  sein  des  FC.  Plus 
particulierement,  un  ensemble  de  donnees  sur  la  structure  des  groupes  professionnels  militaires  de 
l’Aviation  canadienne  a  ete  integre  dans  IPME.  L’etude  menee  documente  les  efforts  deployes  pour 
valider  ce  nouveau  logiciel  de  modelisation.  La  mission  des  engins  telepilotes  (UAV)  modelisee  dans 
IPME  reposait  sur  les  travaux  de  Kobierski  (2004).  L’equipage  du  modele  consiste  en  operateurs  definis 
par  les  enonces  de  taches,  d’habiletes  et  de  connaissances  reliees  au  travail,  etablis  a  partir  des  donnees 
sur  les  groupes  professionnels.  Nous  avons  utilise  un  index  de  similitude  des  emplois  (Farrell  et  al., 
2006),  base  sur  les  donnees  sur  les  taches  integrees  dans  le  modele,  comme  indicateur  pour  predire  la 
performance  des  operateurs.  Pour  confirmer  la  validite  de  cette  approche,  nous  avions  initialement  prevu 
de  reproduire  le  meme  modele  d’engin  pilote  (UAV)  dans  l’outil  IMPRINT  (Improved  Performance 
Research  Integration  Tool)  de  l’Armee  americaine  et  de  comparer  ensuite  les  previsions  en  matiere  de 
performance  produites  par  ces  deux  boites  a  outils  de  modelisation.  Cependant,  comme  IMPRINT  n’ etait 
pas  accessible,  c’est  l’outil  ISMAT  (Integrated  Simulation  Manpower  Analysis  Tool)  qui  a  ete  utilise  a  la 
place.  En  tant  qu’outil  de  modelisation  du  personnel,  ISMAT  a  ete  developpe  essentiellement  pour  cibler 
les  applications  navales.  Par  consequent,  cette  etude  a  surtout  consiste  a  comparer  les  approches  de 
modelisation  du  personnel  differentes  que  sont  IPME  et  ISMAT.  Les  differences  entre  les  deux  sont 
documentees  dans  ce  rapport,  l’accent  etant  mis  sur  les  orientations  futures  de  la  recherche  et  du 
developpement  portant  sur  IPME. 
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Executive  summary 


Integrating  Occupational  Characteristics  into  Human  Performance  Models:  IPME 
versus  ISMAT  Approach 

Christy  Lorenzen,  Alion  Science  &  Technology;  DRDC  CR  2009-059;  Defence  R&D 
Canada  -  Toronto;  August  2009. 

Introduction  or  background:  The  Integrated  Performance  Modelling  Environment  (IPME)  is  a  discrete- 
event  simulation  software  for  developing  computational  human  performance  models.  Prior  to  this  project, 
IPME  lacked  data  for  precisely  representing  military  personnel,  particularly  the  knowledge  and  skills  that 
military  operators  obtained  through  their  professional  training.  A  need  was  identified  to  incorporate 
military  occupational  data  into  IPME  so  that  models  could  be  more  easily  constructed  to  address  issues  in 
the  personnel  and  manpower  domains.  An  applied  research  project  entitled  The  Addition  of  the  Canadian 
Air  Force  Military^  Occupational  Structure  Identification  Code  (MOSID)  into  IPME  was  initiated.  Its 
primary  objective  was  to  expand  IPME’s  modeling  capability  and  address  a  wider  range  of  Human 
Systems  Integration  (HSI)  issues.  Specifically,  this  project  focused  on  the  integration  of  the  Canadian  Air 
Force  occupational  data  into  IPME. 

There  were  four  main  tasks  in  this  project:  (1)  identify  elements  from  the  available  occupational  data  that 
are  relevant  to  computational  human  performance  models;  (2)  link  a  military  occupational  database  to 
IPME;  (3)  apply  occupational  data  in  human  performance  models;  and  (4)  verify  and  validate  this  new 
modeling  capability  by  comparing  it  with  other  modeling  toolkits. 

With  this  project  effort,  analysts  can  now  represent  operators’  MOS-related  professional  knowledge  and 
skills  in  IPME  models.  Thus,  it  is  possible  to  evaluate  whether,  for  example,  operators  are  qualified  to 
perform  an  assigned  task.  In  this  project,  we  explored  the  use  of  a  Job  Similarity  Index  (JSI)  for 
quantitatively  measuring  operator  competency  for  designated  tasks.  This  report  documents  a  validation 
study  that  compared  two  different  methods  for  applying  occupational  data  in  human  performance  models. 

This  study  focused  on  a  contrived  Uninhabited  Aerial  Vehicle  (UAV)  operator  selection  task.  The  newly 
acquired  occupational  (e.g.,  knowledge  and  skills)  data  were  used  both  in  the  IPME  crew  model  as 
operators’  traits,  and  in  the  IPME  task  network  model  for  specifying  task  requirements.  It  allowed 
analysts  to  conduct  a  gap  analysis  and  assess  which  operators  were  better  qualified  to  perform  the 
assigned  tasks.  To  examine  the  validity  of  the  IPME  modeling  process,  the  UAV  model  was  later 
recreated  in  a  modeling  software  application  called  Integrated  Simulation  Manpower  Analysis  Tool 
(ISMAT). 

Results:  The  study  compared  two  ways  to  represent  operator  knowledge  and  skill  traits  in  a  simulation 
model  and  demonstrated  how  such  traits  were  applied  in  operator  qualification  predictions.  Although  both 
represented  acceptable  approaches  for  personnel  modeling,  it  revealed  that  an  integration  of  certain 
ISMAT  features  (e.g.,  the  use  of  a  generic  human  skill  taxonomy)  into  IPME  would  further  enhance 
IPME’s  capabilities. 

Significance:  With  the  addition  of  the  Canadian  Air  Force  occupational  attributes  to  IPME,  analysts 
should  be  able  to  represent  an  operator’s  professional  knowledge  and  skills  in  a  model  and  simulate  Air 
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Force  personnel  more  precisely.  Such  capability  paves  the  way  for  the  use  of  modeling  and  simulation  to 
address  future  personnel  and  staffing  issues. 

Future  plans:  The  contrast  between  IPME  and  ISMAT  revealed  a  significant  gap  between  the  two 
modeling  platforms.  The  study  identified  future  development  needs  for  IPME:  specifically,  the 
introduction  of  generic  human  skill  taxonomy  for  linking  operator  tasks  to  occupational  knowledge  and 
skills.  Such  development  will  reduce  analysts’  effort  in  applying  occupational  attributes  during  model 
construction  and  further  improve  the  representation  of  operator  characteristics  in  human  performance 
models. 
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Sommaire 


Integration  des  caracteristiques  des  groupes  professionnels  dans  des  modeles  de 

performance  humaine  :  I’approche  IPME  et  I’approche  ISMAT 

Par  Christy  Lorenzen;  RDDC  RC  2009-059;  R  &  D  pour  la  defense  Canada  - 
Toronto;  avril  2009. 

Introduction  ou  contexte  :  L’outil  IPME  (Integrated  Performance  Modelling  Environment)  estune 
application  de  simulation  d’evenements  discrets  disponible  sur  le  marche  et  servant  a  developper  des 
modeles  qui  simulent  la  performance  humaine  et  de  systemes.  On  s’est  rendu  compte  qu’il  etait  necessaire 
d’ajouter  au  logiciel  IPME  des  donnees  provenant  de  deux  domaines  MANPRINT  de  l’Armee  americaine 
(MANpower  et  PeRsonnel  INTegration)  -  personnel  et  main-d’oeuvre  -  etant  donne  que  la  sensibilite  de 
IPME  a  l’egard  de  ce  type  de  donnees  a  ete  jusqu’ici  minime.  A  cette  fin,  on  a  done  entrepris  le  projet 
triennal,  L  ’Ajout  du  code  d' Identification  de  la  structure  des  groupes  professionnels  militaire  (IDSGPM) 
dans  IPME,  et  ameliore  cet  outil  en  y  ajoutant  des  modeles  provenant  de  ces  domaines  MANPRINT  dans 
le  but  de  procurer  une  fonction  d’integration  de  systemes  humains  a  la  Force  aerienne  des  FC. 

Comme  il  n’y  avait  pas  de  donnees  sur  le  personnel  dans  les  outils  de  modelisation,  on  a  ajoute  a  IDME 
un  lien  vers  les  donnees  sur  la  structure  des  groupes  professionnels  militaires.  Le  lien  a  permis  aux 
modelisateurs  d’ajouter,  aux  modeles  d’equipage  dans  IPME,  des  niveaux  de  competence  pour  les 
enonces  de  taches,  d’habiletes  et  de  connaissances  reliees  au  travail.  II  a  ainsi  ete  possible  d’evaluer  si  un 
operateur  etait  capable  d’accomplir  la  tache  qui  lui  est  confiee  en  comparant  le  niveau  de  competence 
etabli  pour  cette  tache  au  niveau  d’effort  que  cette  demiere  exigeait.  Le  resultat  de  cette  evaluation 
(calcul)  a  ete  un  index  de  similitude  des  emplois  (ISE). 

On  a  ensuite  entrepris  des  travaux  de  validation  pour  verifier  les  predictions  obtenues  a  l’aide  du  modele 
IPME.  Aux  fins  de  cette  experimentation,  on  a  cree  un  modele  d’engin  teleguide  fonde  sur  une  etude  de 
modelisation  d’engins  teleguides  anterieure.  On  a  fait  appel  au  nouveau  lien  IPME-MOSID  pour 
concevoir  le  modele  d’equipage  et  affecte  les  taches  d’un  niveau  minimum  d’effort  requis.  Les  resultats 
du  modele  permettaient  de  predire,  d’apres  la  valeur  ISE  d’un  operateur,  que  celui-ci  accomplirait  mieux 
la  tache  que  d’autres. 

Initialement,  on  avait  prevu  de  valider  les  resultats  obtenus  au  moyen  d’lPME  en  les  comparant  a  ceux  du 
meme  modele  construit  dans  IMPRINT.  On  en  a  toutefois  ete  empeche  en  raison  de  restrictions  a 
l’exportation  imposees  a  IMPRINT.  Pour  cette  raison,  nous  avons  essaye  de  valider  les  resultats  obtenus  a 
l’aide  d’lPME  en  les  comparant  a  ceux  du  meme  modele  dans  un  autre  outil  de  simulation  appele  ISMAT 
(Integrated  Simulation  Manpower  Analysis  Tool).  En  raison  de  la  difference  intrinseque  qui  existe  entre 
les  deux  outils  de  modelisation,  une  comparaison  directe  des  resultats  du  modele  etait  impossible,  ce  qui 
explique  que  l’etude  a  consiste  essentiellement  a  mettre  en  opposition  les  approches  de  modelisation  du 
personnel  differentes  d’lPME  et  d’lSMAT. 

Resultats  :  L’etude  a  montre  qu’il  y  avait  deux  fatjons  differentes  de  representer  les  connaissances,  les 
habiletes  et  les  caracteristiques  des  operateurs  dans  un  modele  de  simulation  et  de  quelle  maniere  ces 
caracteristiques  etaient  appliquees  aux  predictions  de  la  performance  des  systemes.  Malgre  le  fait  que  les 
approches  de  modelisation  du  personnel  etaient  toutes  deux  des  approches  acceptables,  cette  etude  a 
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revele  que  P  integration  de  certaines  caracteristiques  d’ISMAT,  p.  ex.,  l’utilisation  d’une  classification 
taxinomique  generique  des  qualites  humaines,  bonifierait  davantage  les  fonctionnalites  d’lPME. 

Signification  :  L’ajout  a  IPME  des  attributs  des  groupes  professionnels  militaires  permettra  aux  analystes 
de  representer  les  connaissances  et  les  habiletes  que  les  operateurs  acquierent  dans  le  cadre  de  leur 
instruction  professionnelle.  Le  fait  de  pouvoir  rendre  compte  de  ces  elements  caracteristiques  des 
operateurs  permettra  aux  analystes  d’explorer  la  possibilite  d’utiliser  la  M&S  pour  les  activites  reliees  au 
personnel  et  a  la  dotation. 

Plans  futurs  :  En  permettant  de  comparer  IPME  a  ISMAT,  cette  etude  a  mis  au  jour  l’ecart  qui  existe 
actuellement  entre  ces  deux  jeux  d’outils  de  modelisation  et  a  servi  a  determiner  les  besoins  en  matiere  de 
developpement  fiitur  d’lPME,  en  particulier  l’ajout  d’une  classification  taxinomique  generique  des 
qualites  humaines,  p.  ex.,  l’ensemble  de  competences  Fleishmann.  II  serait  ainsi  possible  de  definir  les 
taches  executees  par  l’homme  (comme  dans  un  modele  de  reseau  de  taches  IPME)  du  point  de  vue  des 
connaissances,  des  competences  et  des  habiletes  requises  pour  les  accomplir.  De  tels  progres 
permettraient  de  faire  avancer  encore  plus  la  representation  des  caracteristiques  humaines  dans  les 
modeles  de  performance. 
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1  Introduction 


The  Integrated  Performance  Modelling  Environment  (IPME)  is  a  commercially-available 
discrete-event  simulation  application  used  for  developing  models  that  simulate  human  and  system 
performance.  The  application  contains  algorithms  for  calculating  operators’  workload  and 
predicting  the  effects  of  stressors  on  performance  [1],  IPME  has  been  a  jointly  developed  by 
Defence  Research  and  Development  Canada  -  Toronto  (DRDC  Toronto),  QinetiQ  Centre  for 
Human  Sciences  (CHS),  and  Alion  Science  and  Technology-MA&D  Operation.  It  is  one  of  the 
tools  used  by  the  Canadian  Air  Force  for  capturing  system  (including  human)  requirements  in 
human  systems  integration  (HSI)  studies. 

A  need  was  identified  to  add  information  to  IPME  from  two  United  States  (US)  Aimy 
MANPRINT  (MANpower  and  PeRsonnel  INTegration)  domains  -  personnel  and  manpower. 

Prior  to  this  effort,  the  sensitivity  of  IPME  to  this  type  of  information  has  been  minimal.  An 
applied  research  project  entitled  The  Addition  of  the  Canadian  Air  Force  Military  Occupational 
Structure  Identification  Code  (MOSID)  for  IPME  was  initiated  in  2005.  Its  purpose  was  to 
enhance  IPME  with  models  from  those  MANPRINT  domains  and  to  improve  the  HSI  capability 
for  the  Canadian  Air  Force.  Note  that  although  the  acronym  MOSID  generally  refers  to  a  5-digit 
occupational  code,  for  convenience  in  this  report,  we  use  the  term  to  refer  to  occupational 
characteristics  that  are  captured  in  occupational  specifications  unless  otherwise  specified.  The 
goals  of  this  project  were  three-fold: 

1.  To  improve  and  increase  the  amount  of  built-in  data  for  analysts  to  use  in  their  IPME 
models; 

2.  To  provide  a  simpler  modeling  method  for  examining  issues  in  the  manpower,  personnel, 
and  training  domains;  and 

3.  To  allow  analysts  to  use  occupational  specifications,  training  data  and  requirements  in 
their  models  to  examine  operator  performance  and  analyse  system  staffing. 

The  addition  of  the  MOSID  to  IPME  will  enhance  the  capability  to  conduct  human-system 
analyses  that  include  occupational  characteristics  of  personnel  (i.e.,  the  operators  or  users  of  the 
equipment).  The  intent  is  that  the  new  capability  resulting  from  this  project  can  be  used  for 
investigating  manning  concepts  for  new  systems,  and  for  supporting  future  capability 
development  and  engineering  process  improvement. 

The  project  consists  of  three  phases.The  first  phase  focused  on  database  creation  and  generated 
two  artifacts:  a  database  that  encapsulated  occupational  characteristics  for  a  selected  Canadian 
Air  Force  occupations  and  a  Java  database  access  application,  CF  Air  Force  MOSID  Application 
(CAMA).  While  this  project  was  conducted,  a  significant  effort  was  made  by  the  Canadian  Forces 
to  modernize  the  military  occupational  structure.  This  effort  was  spearheaded  by  the  MOSART 
project  (Military  Occupational  Structure  Analysis,  Redesign  and  Tailoring).  Outputs  from 
MOSART  included  the  newly  designed  occupational  specifications,  which  provided  the 
foundation  of  occupational  data  that  were  later  incorporated  into  IPME.  Such  Military 
Occupational  Specification  (MOS)  data  were  stored  in  a  relational  database,  independent  of  a 
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modeling  platform  (e.g.,  IPME).  CAMA  allows  users  to  perform  basic  functions  for  managing  the 
relational  database,  such  as  searching  through  the  MOS  database  or  making  data  edits. 

The  second  phase  produced  a  MOSID  plug-in  that  was  integrated  into  IPME  and  improved 
CAMA’s  search  capability.  The  MOSID  plug-in  serves  as  a  bridge  between  the  MOS  data  and 
IPME,  and  allows  an  analyst  to  easily  load  task  and  knowledge  statements  from  the  MOS 
database  into  an  IPME  model.  Such  task  and  knowledge  information  can  also  be  associated  with 
model  tasks  through  task  expression  logic.  When  an  IPME  model  is  executed,  a  Job  Similarity 
Index  (JSI),  (as  described  in  [2]),  can  be  computed  to  estimate  the  level  of  match  between 
operators  with  specific  skill  sets  and  their  designated  tasks. 

The  third  phase  of  the  project  involved  a  study  that  contrasted  two  different  modeling  approaches 
for  addressing  manpower  issues.  An  Uninhabited  Air  Vehicle  (UAV)  operator  selection  task  was 
used  as  a  test  case.  An  IPME  UAV  model,  created  previously  for  a  separate  project  [4],  was  used 
as  a  baseline  and  further  modified  by  incorporating  occupational  data.  This  model  was  later 
translated  into  the  Integrated  Simulation  Manpower  Analysis  Tool  (ISMAT).  The  different 
methods  used  in  IPME  and  ISMAT  for  processing  occupational  data  were  compared  and  reported 
in  this  document. 

Table  1  provides  a  detailed  chronological  description  of  key  project  activites. 


Table  1:  The  Addition  of  the  CF  AF  MOSID  into  the  IPME  Project  Tasks 


Year  1 

1.  Communicate  with  Human  Resources  -  Military  (HR(mil))  and  coordinate  with 
the  Military  Occupational  Structure  Analysis,  Redesign,  and  Tailoring  (MOSART) 
project. 

2.  Design  and  populate  a  database  with  MOS  data.  Document  the  database  design. 

3.  Design  and  create  a  user  interface  called  the  CF  AF  MOSID  Application  (CAMA) 
to  view  and  edit  the  MOS  database. 

Year  2 

1.  Update  CAMA  with  new  data  received  from  MOSART. 

2.  Coordinate  with  the  Canadian  Forces  Experimentation  Centre  (CFEC)  on  the 
Uninhabited  Aerial  Vehicle  (UAV)  crew  selection  study  and  prepare  the  scenario  for 
the  validation  experiment. 

3.  Under  the  guidance  of  the  Scientific  Authority  (SA),  create  an  experiment  plan  for 
validating  IPME’s  newly  added  capability. 
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Year  3 


1.  Add  the  capability  to  IPME  to  access  the  MOSID  database  and  incoiporate  the 
data  in  a  model. 

2.  Create  a  UAV  model  for  a  crew  selection  task  using  IPME. 

3.  Validate  IPME’s  new  capability  by  contrasting  the  UAV  model  prediction  to  the 
results  obtained  in  the  CFEC  study. 

4.  Create  the  same  UAV  model  in  the  Integrated  Simulation  Manpower  Analysis 
Tool  (ISMAT). 

5.  Compare  how  IPME  and  ISMAT  use  personnel  and  manpower  data  in  terms  of  the 
UAV  models  that  were  created.  Compare  results  from  these  two  models. 


Work  on  this  project  began  with  Military  Occupational  Code  (MOC)  documents  received  from 
the  Scientific  Authority  (SA).  These  documents  allowed  the  design  and  population  of  a  prototype 
database  using  Microsoft  Access,  which  formed  the  basis  of  the  initial  version  of  CAM  A.  Table  2 
shows  the  list  of  Air  Force  occupations  included  in  this  MOC  database. 


Table  2:  Initial  Set  of  Occupations  Included  in  the  Prototype  Database 


Occupation 

MOC 

MOSID 

Aviation  Systems  Technician 

514 

00135 

Air  Navigator 

031 

00182 

Aerospace  Control 

039 

00184 

Airfield  Engineering 

046 

00189 

Aerospace  Control  Operator 

168 

00337 

These  MOC  documents  contain  occupational  identifiers  and  descriptions  of  occupational 
classifications,  working  conditions,  career  development  plans,  and  job  performance  requirements. 
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Such  information  is  frequently  used  for  identifying  occupational  training  requirements,  guiding 
human  resource  management,  providing  information  for  potential  recruits,  and  coding,  recording, 
and  reporting  occupational  qualifications. 

The  CF  underwent  an  initiative  to  modernize  its  occupational  classification  system  through  the 
MOSART  project.  It  was  initiated  by  HR(mil)  with  an  objective  to  develop  a  military 
occupational  structure  that  enhances  the  strategic  capability  of  the  CF  to  meet  an  increasing  array 
of  operational  missions  with  highly  skilled  and  motivated  people,  while  expanding  the  range  of 
career  opportunities  for  CF  personnel  and  maximising  the  efficiency  of  the  CF  Human  Resources 
(HR)  Management  System.  Due  to  the  close  link  between  MOSART  and  this  project,  a  brief 
description  of  MOSART  is  provided. 

Specifically,  the  MOSART  project  had  three  primary  goals: 

1.  To  establish  a  new  occupational  structure  that  enhances  operational  capabilities,  is  cost 
effective,  and  contributes  to  increased  retention  in  the  CF  through  the  use  of  broader 
career  fields; 

2.  To  revise  all  supporting  policies  and  procedures  related  to  the  military  occupational 
structure;  and 

3.  To  create  a  master  implementation  plan  that  depicts  the  transition  to  the  new  structure 
across  the  CF. 

To  achieve  these  goals,  MOSART  adopted  an  evolutionary  approach  towards  the  required  MOS 
modification,  and  started  with  an  examination  of  the  overall  CF  work  requirement  at  the  job  level. 
Traditionally,  the  development  of  work  requirements,  as  well  as  the  process  for  reviewing  and 
updating  the  MOS,  has  taken  place  using  occupations  as  the  basic  building  block.  The  use  of  the 
occupational  model  has  essentially  created  a  stove-pipe  effect,  and  has  necessitated  the 
establishment  and  maintenance  of  separate  and  detailed  occupational  specifications  for  both  the 
Regular  Force  and  the  Primary  Reserve.  Having  recognized  this  problem,  MOSART  initially 
analyzed  and  defined  the  CF  work  at  a  job  level.  Jobs  serve  as  the  basic  entities  for  managing  CF 
personnel.  Once  jobs  are  identified  and  defined,  decisions  can  be  made  on  how  to  group  similar 
jobs  into  larger  constructs  in  the  MOS,  for  example,  occupations,  sub-occupations,  and  career 
fields. 
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Military  Occupational  Structure 


Career  Fields 


organized  into  various  MOS  constructs 


Figure  1:  An  illustration  of  the  CF  military  occupational  structure. 

The  MOSART  project  established  a  logical  and  sequential  process  to  achieve  the  desired  MOS 
structure,  which  included  amending  existing  specifications  with  their  inherent  work  requirements 
and  their  associated  impacts  on  HR  management  areas.  A  modernized  MOS  makes  it  possible  to 
transition  the  HR  management  system  from  the  old  position-based  system  to  the  new  job-based 
system.  Currently,  the  CF  manages  its  work  through  approximately  85,000  positions.  Following 
the  principle  of  job-based  management,  these  positions  are  consolidated  into  8,000  to  10,000 
military  jobs  based  on  their  work  requirements.  This  allows  for  a  more  effective  and  efficient  way 
to  manage  personnel  across  the  CF. 

Another  important  endeavour  in  the  MOSART  project  was  the  establishment  of  links  between 
jobs  and  their  competency  requirements.  This  produces  a  robust  system  that  allows  the 
development  of  succession  plans,  career  paths,  and  expeditious  evaluation  of  CF  members  against 
their  designated  jobs.  At  the  strategic  level,  this  system  assists  in  the  development  of  integrated 
professional  development  strategies  and  plans.  For  the  CF  members,  the  system  is  more  open  and 
transparent,  allowing  them  to  gauge  their  personal  qualifications  and  competencies  against 
existing  job  inventories,  thus  providing  a  clearer  picture  of  professional  development 
requirements,  personal  goals,  and  in  turn,  more  control  and  influence  over  individual  careers. 

During  the  process  of  defining  the  CF  work  domain,  MOSART  started  with  a  high-level  gap 
analysis  to  identify  key  discrepancies  among  CF  strategic  capabilities,  force  structure,  and 
occupational  specifications.  A  standardized  web-based  survey  was  used  in  job  analysis  for 
developing  occupational  specifications.  Besides  the  existing  tasks,  skills  and  knowledge  (TSK) 
inventory,  competency  measures  were  collected  in  the  survey  and  incorporated  into  the  job 
description.  The  re-grouped  occupations  were  coded  using  a  new  five-digit  MOSID,  which 
replaced  the  three-digit  MOC  used  in  the  old  MOS. 
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Accompanying  this  change  of  the  coding  system,  significant  modification  was  also  made  within 
each  occupational  specification.  In  the  context  of  this  project,  occupational  specifications  were 
the  primary  data  source  for  the  integration  effort.  When  the  project  was  conducted,  MOSART  had 
not  completed  the  job  analysis,  therefore,  the  occupational  specifications  that  were  used  in  this 
project  were  provisional  in  nature.  The  analysis  described  in  this  report  reflects  our  best 
knowledge  obtained  from  the  latest  version  of  the  occupational  specifications  at  the  time  the 
study  was  performed. 

As  the  second  year  of  the  project  began,  a  MOS  database  (which  we  named  for  internal  use 
“version  1/July  2006”)  was  received.  The  new  database  adopted  a  new  data  structure,  as  a  result, 
CAMA  was  updated  accordingly.  In  addition,  CAMA  was  modified  to  enhance  its  data  searching 
and  editing  capability.  A  MOSID  plug-in  to  IPME  with  an  enhanced  search  capability  was  also 
developed  during  this  phase. 

While  the  database  and  application  development  were  taking  place,  we  also  continued  to  work 
with  the  Canadian  Forces  Experimentation  Centre  (CFEC)  and  MOSART.  Reports  and  data  from 
the  UAV  Crew  Selection  study  [2]  were  obtained  from  CFEC,  and  were  later  extensively 
leveraged  in  the  follow-up  validation  study  that  took  place  in  the  final  phase  of  the  project. 

This  report  is  a  summary  of  the  validation  effort.  In  the  following  sections,  the  report  describes: 
the  problem  statements  (Section  2),  two  software  platforms  (Section  3),  typical  modeling 
processes  (Section  4),  a  UAV  model  created  in  both  IPME  and  ISMAT  (Sections  5  and  6, 
respectively),  and  a  final  summary  of  findings  (Section  7). 
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2  Problem  Statement 


Alion  Science  and  Technology  (formerly  Micro  Analysis  &  Design,  Inc.)  began  the  development 
of  IPME  in  1995  with  QinetiQ  CHS.  In  1998,  DRDC  Toronto  joined  the  development  program. 
Together  with  DRDC  Toronto  and  QinetiQ,  Alion  designed  this  discrete-event  simulation 
application  to  support  human  factors  analyses,  more  specifically,  to  support  examination  of 
stressors  on  human  performance.  IPME  focuses  on  the  human,  the  tasks  that  the  human  performs 
in  support  of  a  goal,  the  environment  in  which  the  human  operates,  the  stressors  that  affect  human 
performance,  and  an  interface  with  external  simulations.  A  secondary  goal  of  the  development 
program  is  to  offer  modeling  flexibility  to  the  analyst.  This  is  realized  through  plug-and-play 
component  models,  user-defined  functions,  performance  shaping  functions  (PSFs),  and  a 
configurable  master  database. 

Since  development  has  focused  on  functionality  and  flexibility,  built-in  models  packaged  with 
IPME  are  minimal.  IPME  does  not  include  actual  data  or  populated  PSFs,  environment  models,  or 
operators.  While  the  functionality  is  available  for  analysts  to  add  customized  data,  the  process  can 
be  time-consuming.  A  need  was  recognized  to  provide  readily-available,  built-in  data  to  support 
model  creation  and  analysis.  In  particular,  two  objectives  were  identified  for  IPME: 

1.  Built-in  data  to  support  model  construction. 

2.  A  simpler  method  for  developing  models  in  support  of  the  personnel  and  training 
domain. 

While  it  is  possible  to  examine  personnel  characteristics  and  training  aspects  of  a  system  using 
IPME,  the  job  analysis  is  tedious  because  of  the  lack  of  built-in  data.  Operator  occupational 
characteristics  and  training  requirements  are  important  aspects  of  system  modeling;  therefore,  it  is 
desirable  to  enhance  direct  support  to  these  types  of  models  in  IPME. 

The  US  Army  has  long  recognized  the  importance  of  including  training  data  when  examining  a 
system.  The  US  Army  MANPRINT  program  started  in  the  mid-1980s  with  a  goal  of  optimizing 
“ how  well  new  systems  work  with  soldiers  in  the  field  by  including  the  soldier  in  the  design 
process ”  [7].  By  examining  how  soldiers  can  perform  their  jobs  with  provided  equipment  and 
taking  into  account  human  characteristics  such  as  aptitude  and  physical  traits,  combat  readiness  is 
increased.  There  are  seven  MANPRINT  domains  that  are  considered  during  the  system 
acquisition  process: 

1.  Personnel  Capabilities:  the  cognitive  and  physical  abilities  of  a  soldier. 

2.  Manpower:  the  number  of  personnel  required  to  operate,  maintain,  sustain,  and  provide 
training  for  systems  [6]. 

3.  Training:  education  required  to  provide  personnel  the  knowledge  and  skills  they  need  to 
perform  their  jobs. 
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4.  Human  Factors  Engineering  (HFE):  the  integration  of  human  characteristics  into  system 
definition,  design,  development,  and  evaluation  to  optimize  human-machine  performance 
under  operational  conditions  [6]. 

5.  System  Safety  (SS):  the  variables  that  minimize  the  potential  for  system  errors  (human  or 
machine). 

6.  Health  Hazards  (HH):  the  system  variables  that  increase  the  risk  of  human  injury  or 
death. 

7.  Soldier  Survivability  (SSv):  the  system  variables  that  reduce  fratricide,  detectability,  and 
probability  of  being  attacked  [6],  and  minimize  system  damage  and  operator  fatigue. 

Many  of  the  MANPRINT  domains  can  be  modeled  in  IPME  via  the  component  model  structure. 
By  incorporating  the  Canadian  Air  Force  MOSID  data  in  IPME,  the  analyst  will  be  able  to  use  the 
occupational  specifications,  training  data  and  requirements  in  their  models  to  examine  operator 
performance  and  analyze  system  staffing. 

MOSID  data  were  reviewed  and  initially  converted  into  a  Microsoft  Access  database.  CAMA  was 
then  created  for  managing  this  database.  Its  basic  functions  included  MOS  data  viewing,  key 
word  searching,  and  data  editing.  CAMA’s  capability  was  later  integrated  into  a  MOSID  plug-in 
that  could  be  accessed  from  IPME,  which  enabled  analysts  to  specify  MOSIDs  for  operators  in  a 
crew  model. 

During  the  course  of  the  project,  the  MOS  database  was  upgraded  several  times,  each  associated 
with  a  change  of  its  internal  data  structure.  As  a  result,  we  updated  CAMA  and  the  IPME  plug-in 
accordingly.  Since  the  MOS  data  had  a  big  impact  on  both  application  development  and  model 
construction,  a  detailed  review  of  these  data  is  provided  in  the  next  sub-section. 


Table  3:  Nine  Occupations  Initially  Reviewed 


Occupation 

MOSID 

Airborne  electronic  sensor  operator 

00019 

Flight  engineer 

00021 

Aviation  systems  technician 

00135 

Avionic  systems  technician 

00136 

Air  navigator 

00182 
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Aerospace  control  officer 

00184 

Aerospace  engineering 

00185 

Airfield  engineer 

00189 

Aerospace  control  operator 

00337 

2.1  Occupational  Data  and  the  Review  Process 

This  section  describes  the  available  occupational  data  and  the  process  used  to  review  and  interpret 
the  data. 


MOSIDl 
Job  Requirements 
Minimum  Entry  Requirements 
Related  Civilian  Occupations 
Core  Occupational  Jobs 
Employment  and  Training 


Force  1 

Language  Requirement 
Security  Clearance 

Subdivisionl 

Tasks 

Skills 

Knowledge 

Subdivision_2 

Tasks 

Skills 

Knowledge 

Force_2 

Language  Requirement 
Security  Clearance 


Figure  2:  MOS  Data  Relationships  based  on  the  Initial  Review  of  Occupational  Specifications 

Initially,  the  specifications  for  nine  Canadian  Air  Force  occupations  (see  Table  3)  were  reviewed 
to  determine  what  data  could  be  used  in  human  performance  models.  These  specifications  were  in 
Microsoft  Word  document  format.  They  were  reviewed  multiple  times  by  two  analysts  to 
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determine  the  data  structures  and  relationships.  The  purpose  was  to  import  these  data  into  a 
database  and  identify  information  elements  that  can  be  applied  in  a  computational  model. 
Generally  speaking,  an  occupational  specification  contains  information  about  job  requirements, 
training  needs,  and  career  progression  plans.  Each  occupation  is  typically  classified  into  three 
different  force  types  (Regular  Force,  Primary  Reserve,  and  Special  Force),  each  of  which  further 
consists  of  several  subdivisions.  Job  tasks,  skill  requirements,  and  knowledge  requirements  are 
detailed  at  the  subdivision  level.  Requirements  filter  down  the  hierarchy.  For  example,  minimum 
entry  requirements  are  applicable  for  all  forces  and  subdivisions  of  any  occupation.  Security 
clearance  and  language  requirements  are  specific  to  forces  and  their  subdivisions.  Figure  2  is  a 
conceptual  diagram  that  displays  our  initial  interpretation  of  the  hierarchical  relationships  in  the 
MOS  data. 


MOSID2 
Job  Requirements 
Minimum  Entry  Requirements 
Related  Civilian  Occupations 
Core  Occupational  Jobs 
Employment  and  Training 

Subdivisionl 

Force 

Security  Clearance 

Tasks 

Skills 

Knowledge 

Subdivision_2 

Force 

Security  Clearance 

Tasks 

Skills 

Knowledge 


Figure  3:  Updated  MOSID  Relationships 

The  second  batch  of  MOS  data  was  delivered  to  us  in  the  form  of  four  Microsoft  Access 
databases.  Compared  with  the  occupational  specification  documents  that  we  reviewed  previously, 
these  databases  presented  several  challenges.  First,  the  MOS  structure  in  the  new  databases 
differed  from  the  one  in  the  specification  documents.  Specifically,  the  data  hierarchy  was 
reduced.  Figure  3  illustrates  the  modified  data  relationships.  Note  that  the  force  designation  and 
security  clearance  information  are  now  included  in  the  subdivision  (“Job”).  Fanguage 
requirements  are  specified  as  knowledge  statements  for  a  subdivision.  Upon  further  examination 
of  the  database  tables,  we  noticed  that  several  data  keys  and  enumerated  type  variables  were 
assigned  different  values  across  the  databases.  This  prevented  us  from  following  the  original  plan 
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to  combine  four  databases  into  a  single  consolidated  one.  Although  using  a  single  database  was 
desirable  because  it  would  simplify  data  management  and  information  access,  this  artifact  in  data 
source  forced  us  to  choose  an  less-than-optimal  solution.  By  presenting  occupational  data  in 
separate  databases  (one  for  each  occupation),  analysts  were  limited  to  using  the  data  from  a  single 
occupation  at  a  time.  The  provisional  nature  of  the  source  data  caused  functional  and  usability 
limitations.  With  the  completion  of  MOSART,  structural  discrepencies  across  MOS  data  will  be 
eliminated  and  we  anticipate  this  issue  will  be  resolved. 

The  database  management  tool,  CAMA,  was  developed  to  access  multiple  MOS  databases.  It 
includes  a  user  interface  for  specifying  the  data  source.  After  an  occupational  database  is  selected, 
corresponding  MOSIDs  in  this  database  are  listed  on  the  left-hand  side  (see  Figure  4).  In  this 
example,  after  an  air  navigator  occupation  is  selected  (as  highlighted  in  the  left  pane),  its 
corresponding  jobs  are  automatically  shown  on  the  right-hand  side,  which  includes  an  air 
mobility  navigator  and  a  maritime  tactical  control  officer  (only  these  two  jobs  were  populated 
with  data  when  this  study  was  conducted).  More  information  about  each  job  can  be  further 
revealed,  including  its  duty  and  responsibility  descriptions,  working  environment  conditions,  task, 
skill,  and  knowledge  statements,  and  performance  requirements. 


Figure  4:  CAMA  Main  Dialogue 

Some  of  the  occupational  data  are  textual  descriptions,  as  shown  in  the  following  example  for  an 
air  mobility  instructor  navigator  (job  162  from  the  MOSID  00182  -  Air  Navigator)  [5]. 


Environment:  “The  Maritime  Helicopter  Instructor  Navigator  performs  his  tasks  either  on 
the  ground  or  in  the  air  in  the  CH-124  Sea  King.  On  the  ground,  he  will  work  in  a 
classroom  environment,  a  simulator  or  an  office,  normally  located  near  the  flight  line. 
This  job  applies  to  Development  Period  2.” 

Physical  Effort:  “The  incumbent  is  engaged  in  heavy  physical  activities  of  various  kinds. 
The  physical  effort  required  for  lifting,  pulling,  and  similar  activities  is  considerable. . .” 
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Others  have  quantitative  ratings,  such  as  the  task,  skill,  knowledge  (TSK)  requirements.  For  each 
occupation,  its  required  TSKs  are  first  specified  in  the  form  of  statements.  Each  statement  is  then 
associated  with  a  numerical  rating  obtained  on  a  5 -point  proficiency  scale  (see  Table  6). 

Examples  of  a  task  and  knowledge  statement  are  shown  inTable  4.  Note  that  in  the  databases  we 
obtained  from  MOSART,  the  skill  statements  were  not  populated  for  any  of  the  jobs.  This  artifact 
forced  us  to  use  task  and  knowledge  data  in  the  following  validation  effort.  However,  it  is 
expected  that  the  skill  data  can  be  applied  in  an  IPME  model  exactly  the  same  way  as  the  TK 
data,  therefore,  this  artifact  does  not  limit  the  generalizability  of  the  validation  effort. 

Table  4:  Sample  Task  and  Knowledge  Statements 


Task  Statement 


Duty  ID 

C  -  Navigation 

Serial 

T0082 

Statement 

Program  navigation  system  equipment 

Proficiency  Level 

4 

Knowledge  Statement 


Duty  ID 

A  -  Air  Traffic  Control 

Serial 

K0001 

Statement 

ATC  Manual  of  Operations 

Proficiency  Level 

2 

Since  only  quantitative  data  can  be  used  in  a  computational  model,  the  next  step  in  our  analysis 
was  to  determine  what  data  could  be  incorporated  into  IPME.  Table  5  shows  the  typical  structure 
of  an  occupational  specification.  Based  on  our  analysis,  we  classified  such  a  specification  into 
three  categories  based  on  the  information’s  relevancy  to  IPME  modeling. 

Category  1  (highlighted  in  red)  refers  to  data  that  are  irrelevant  to  a  computational  human 
performance  model.  Examples  of  such  data  are  pay  codes  and  occupational  development  plan. 

Category  2  (highlighted  in  yellow)  indicates  data  that  are  informative  for  a  human  performance 
model,  however,  their  current  form  does  not  support  a  direct  linkage  to  IPME  modeling. 
Specifically,  these  data  are  mostly  textual  descriptors  and  consists  of  descriptions  of  working 
conditions,  physical  and  mental  efforts.  The  lack  of  numerical  representation  makes  it  difficult  to 
incorporate  the  data  directly  in  an  IPME  model. 

On  the  other  hand,  these  data  do  provide  analysts  with  information  about  the  nature  of  work  in 
which  operators  typically  engage.  They  can  be  used  indirectly  in  IPME  to,  for  example,  support 
the  identification  of  environmental  stressors  or  the  formulation  of  performance  shaping  functions. 
Consequently,  we  allowed  analysts  to  access  these  data  through  the  MOSID  plug-in  and  left  it  as 
the  analysts’  responsibility  to  develop  numerical  relations  for  these  data  in  an  IPME  model. 

Category  3  (highlighted  in  green)  represents  data  that  can  be  directly  applied  in  IPME  models. 
More  specifically,  these  are  task,  skill,  and  knowledge  (TSK)  statements,  and  their  associated 
profeciency  requirement  scores.  Together,  they  represent  characteristics  that  military  members 
have  acquired  through  their  professional  training.  The  task  statements  describe  the  activities  in 
which  an  operator  from  a  particular  trade  is  likely  to  engage,  and  a  numerical  score  is  provided 
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for  each  task  statement  to  indicate  the  required  performance  level.  Similarly,  knowledge  and 
skills  required  to  conduct  these  tasks  are  specified  in  KS  statements  and  a  rating  score  is  provided 
on  the  same  proficiency  scale  (see  Table  6).  The  numerical  nature  of  the  proficiency  requirement 
score  makes  the  TSK  our  primary  target  for  this  integration  effort. 

Table  5:  A  Structural  Breakdown  of  a  Typical  Occupational  Specification 


Occupational  Specification 


Section  1 : 
General 


Section  2:  Occupational  Section  3:  Occupational 

Development  Performance  Requirements 


Sub¬ 

divisions 

Occupational  Specialty 
Specification 


Job 

Requirements 

Additional  Performance 
Requirements 

Comprehension  and 
Judgment 

Occupational  Training 
and  Experience 

Working  Conditions 
Responsibility 
Resources 
Services 
Consequences 
of  Error 
Environment 
Hazards 
Job  Effort 


Qualification  Levels 
Duty  Areas  and  Task 
Statements 


Skill  and  Knowledge 
Statements 

Performance  Requirement 
Matrix 


Core  Occupational  Jobs 
Integrated  Operational 
Framework 


Physical  Effort 
Mental  Effort 
Analysis  Effort 


Modeling 

essential 

Informational 

Irrelevant 
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Table  6:  Proficiency  scale  used  for  rating  Task,  Skill,  and  Knowledge  (TSK)  statements: 


LEVEL 

TASK/SKILL 

KNOWLEDGE 

1 

the  level  of  proficiency  required 
to  perform  parts  or  elements  of 
duties  and  tasks  under  continuous 
supervision 

an  awareness  of  the  basic  definitions  and 
concepts  associated  with  a  topic  or  a  body 
of  knowledge 

2 

the  level  of  proficiency  normally 
required  to  perform  duties  and 
tasks  under  normal  supervision 

the  level  of  understanding  of  definitions  and 
basic  concepts  which  enables  the  relating  of 
this  knowledge  to  job  requirements 

3 

the  level  of  proficiency  required 
to  independently  and  safely 
perform  duties  and  tasks 

the  level  of  understanding  of  theory  and 
principles  of  a  topic  or  body  of  knowledge 
that  is  usually  gained  through  formal 
training  and  job  experience  and  which 
enables  critical  thought  and  independent 
performance 

4 

the  level  of  proficiency  which 
usually  can  be  acquired  by 
considerable  training  and 
extensive  practical  job 
experience 

the  level  of  knowledge  which  is  gained  from 
formal  training  and  education  and 
considerable  job  experience.  This 
knowledge  enables  the  synthesis/integration 
of  theory  facts  and  practical  lessons  learned 
to  support  the  identification  of  solutions  to 
non-routine  problems 

5 

the  level  of  proficiency  indicated 
by  a  mastery  of  techniques  and 
expert  application  of  procedures 

a  recognized  level  of  expertise,  which 
includes  a  mastery  of  theory  and 
application,  related  to  a  given  body  of 
knowledge 

There  are  two  potential  ways  to  apply  TSK  data  in  an  IPME  simulation  model.  Firstly,  the  crew 
model,  where  operators  are  defined,  allows  an  analyst  to  specify  a  military  occupation  for  each 
operator  by  selecting  the  appropriate  occupational  code  from  the  MOS  database.  Once  an 
occupation  is  specified,  the  relevant  skills  and  knowledge  statements,  together  with  their 
proficiency  requirements,  can  automatically  populate  the  occupational  traits  in  the  operator 
model.  Figure  5  shows  the  dialogue  in  IPME  with  the  knowledge  statement  “ATC  Manual  of 
Operations”  selected.  A  trait  representing  that  knowledge  statement  is  created,  and  the  trait  mean 
value  is  set  to  the  proficiency  value  of  the  knowledge  statement.  Knowledge  statements  are  added 
to  IPME  as  traits  because  IPME  traits  can  be  sampled  from  a  selected  distribution.  In  this  case, 
sampled  traits  could  represent  varying  degrees  of  operator  proficiency. 
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&  Edit  Operator:  MC 


Figure  5:  Adding  MOSID  Data  to  an  IPME  operator 

Secondly,  the  task  statements  are  integrated  into  the  IPME  task  network  models  via  a 
modification  of  the  task  definition,  which  allows  analysts  to  connect  a  particular  human  activity 
to  one  or  more  task  statements  defined  in  the  occupational  specifications.  This  connection  is 
accomplished  by  categorising  the  activity  into  a  relevant  duty  area,  then  selecting  the  most 
representative  task  statements  and  their  associated  performance  requirements. 

After  the  task  has  been  assigned  to  a  particular  operator,  it  is  possible  to  compare  the  list  of 
required  task  statements  (TSreq)  to  the  list  of  task  statements  described  in  operator’s  occupational 
specification  (TSop).  A  job  similarity  index  (JSI)  can  be  calculated  by  comparing  the  level  of 
matching  between  TSreq  and  TSop.  A  JSI  score  of  one  means  that  the  designated  operator 
possesses  all  the  skills  and  knowledge  that  are  required  by  the  operational  tasks;  whereas  a  score 
of  zero  represents  the  opposite  extreme  where  the  operator  has  none  of  the  required  skills  and 
knowledge.  Such  an  index  is  typically  used  in  cross-occupation  comparisons.  One  assumption  is 
that  the  occupation  that  produces  a  higher  JSI  score  better  satisfies  the  job  requirements,  and 
therefore  superior  work  performance  can  be  achieved  by  assigning  an  operator  from  this 
occupation  to  the  task.  For  more  information  about  JSI  and  its  application,  see  [2]. 
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3  Introduction  to  IPME  and  ISMAT 


The  intent  of  this  validation  effort  was  to  apply  the  newly  developed  IPME  capability,  create 
simulation  models  that  utilize  occupational  data,  and  examine  the  model’s  sensitivity  to  these 
occupational  characteristics.  Since  the  US  Army’s  IMPRINT  software  implemented  similar 
functionality,  the  original  project  plan  was  to  compare  these  two  modeling  packages.  However, 
because  IMPRINT  was  unavailable  due  to  US  export  restrictions,  ISMAT  was  used  as  a 
replacement.  This  has  several  implications,  the  most  significant  of  which  is  that  ISMAT  consists 
of  occupational  data  from  the  US  Navy,  and  because  of  the  significant  difference  between  the  Air 
Force  and  Navy  occupations,  it  is  difficult  to  directly  compare  model  outputs  from  ISMAT  and 
IPME.  Since  ISMAT  adopts  a  drastically  different  approach  for  representing  and  utilizing 
occupational  traits,  it  was  decided  to  focus  on  comparing  the  modeling  methodologies.  A  brief 
introduction  to  these  two  modeling  packages  is  provided  in  this  section. 

3.1  IPME 

IPME  is  an  integrated  modeling  environment  for  representing  human  activities  and  analyzing 
system  performance.  An  IPME  model  is  organized  as  a  system,  which  is  a  collection  of  models 
and  data  that  represent  human  operators,  their  tasks,  and  the  operational  environment.  A  system  is 
comprised  of  component  models,  including  environment,  crew,  performance  shaping  functions, 
task  network  models,  and  a  measurement  suite.  Operator  activities  are  mapped  out  in  a  network  of 
tasks  (i.e.,  a  task  network  model).  The  operators’  characteristics,  such  as  physical  properties 
(e.g.,  weight,  height),  traits  (e.g.,  fitness,  training),  and  states  (e.g.,  boredom,  hunger)  are 
represented  in  a  crew  model.  The  environmental  stressors,  e.g.,  physical  conditions,  crew, 
mission,  or  threat  factors,  are  captured  in  an  environment  model.  The  impact  of  stressors  on 
system  performance  is  defined  in  a  performance  shaping  function  (PSF)  model.  The  measurement 
suite  supports  comprehensive  data  collection. 

IPME  provides  a  graphical  interface  for  describing  system  processes  in  a  task  network  model. 

The  task  network  model  contains  tasks  and  linkages  between  those  tasks.  Tasks  generally  denote 
human  activities,  but  can  also  be  used  to  support  the  simulation.  An  operator  task  contains  timing 
information,  conditions  for  execution,  and  operator  assignments.  Operators  from  the  crew  model 
may  be  statically  assigned  to  particular  tasks,  or  they  may  be  dynamically  assigned  to  tasks  based 
on  operator  availability,  aptitude,  or  other  user-defined  criteria.  Logical  expressions  can  be 
defined  inside  a  task  for  manipulating  the  system  state  while  a  model  is  executed.  Resources  can 
be  specified  for  operator  tasks  which  are  used  by  IPME’s  internal  algorithm  for  predicting  the 
operator’s  workload.  The  linkages  between  those  tasks  represent  system  processes. 

While  it  is  optional  to  use  the  environment,  crew,  and  PSF  models,  using  all  three  in  combination 
with  the  task  network  model  takes  advantage  of  the  component  models’  plug-and-play  feature. 

For  example,  different  crew  models  can  be  combined  with  a  single  task  network  model  to  allow 
an  analyst  to  examine  how  different  operators  would  perform  the  same  set  of  tasks.  The 
relationship  between  these  component  models  is  shown  in  Figure  6. 
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Figure  6:  Component  Model  Relationship 


In  addition,  IPME  allows  analysts  to  create  a  measurement  suite,  which  provides  a  simple  method 
for  defining  an  experiment  and  modifying  system  variables  in  a  structured  manner.  For  example, 
a  measurement  suite  can  be  used  to  sample  operator  traits  and  anthropometry  values  from 
different  distributions  across  simulation  runs. 

In  this  project,  IPME  was  further  developed  to  include  a  plug-in  for  easy  incorporaton  of  job- 
related  data  from  the  MOS  database.  A  tab  was  added  to  the  operator  definition  interface  in  the 
crew  model  to  allow  users  to  filter  and  select  information  from  the  database.  This  enables  analysts 
to  choose  specific  job-related  data  and  incorporate  that  data  into  a  model  as  operator 
characteristics.  A  sample  screenshot  of  the  modified  interface  is  shown  in  Figure  7. 


Figure  7:  Edit  Operator  Dialog 


As  part  of  this  project,  CAMA  was  created  as  a  companion  tool  to  support  the  use  of  occupational 
data  IPME.  It  provides  a  graphical  interface  for  analysts  to  view,  modify,  and  search  the  MOS 
data.  In  CAMA,  occupational  data  from  different  MOS  databases  can  be  viewed  in  a  hierarchical 
structure.  The  hierarchy  reflects  the  relationships  between  occupational  fields  and  jobs.  When  an 
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occupation  (e.g.,  a  MOSID)  is  selected,  the  corresponding  jobs  are  automatically  ordered  and 
listed  (as  shown  previously  in  Figure  4).  Data  can  be  modified  for  each  occupation  or  job.  Figure 
8,  as  an  example,  is  the  interface  for  modifying  the  basic  occupational  definition.  In  order  to 
prevent  accidental  modification,  the  data  are  presented  to  the  user  in  read-only  format.  The  user 
must  click  a  button  before  changes  can  be  made  to  the  database. 


Figure  8:  Interface  for  Modifying  the  Basic  Occupational  Definition 


At  the  job  level,  data  are  entered  on  a  series  of  tabbed  dialogs,  as  shown  in  Figure  9.  Details 
regarding  the  job  description,  environment,  requirements  and  task,  skill  and  knowledge 
statements  can  be  modified  on  the  tabbed  dialogs.  Such  edits  are  immediately  stored  in  the  MOS 
database. 


Figure  9:  Job-related  Data 


CAMA  supports  keyword  searches.  This  provides  analysts  a  quick  way  to  navigate  through  a 
large  amount  of  occupational  data  and  to,  for  example,  identify  occupations  that  possess  certain 
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skill  sets.  Search  feedback  is  presented  in  a  hierarchical  arrangement  with  the  lowest  level 
showing  the  context  of  the  keyword.  An  analyst  can  then  double  click  each  feedback  entry  and 
the  relevant  information  from  that  location  in  the  database  will  be  presented.  Sample  results  from 
a  keyword  search  are  shown  in  Figure  10. 


Figure  10:  Sample  Keyword  Search  Results  in  CAMA 


3.2  ISMAT 

ISMAT  was  developed  under  a  Small  Business  Innovative  Research  (SBIR)  project  entitled 
Human  Systems  Integration  (HSI)  Rapid  Analysis  Tool  for  Evaluation  of  System  Concepts  Early 
in  Development,  and  was  sponsored  by  the  Naval  Surface  Warfare  Center-Dahlgren  Division 
(NSWC-DD). 

ISMAT  was  created  to  support  planning,  designing,  and,  most  importantly,  evaluating  alternative 
manning  and  automation  concepts  prior  to  implementing  technology,  with  the  goal  of  reducing 
crew  sizes,  workload,  or  cost.  This  tool  is  designed  to  be  used  by  analysts  to  evaluate  operator 
workload  and  cost  for  different  manning  options.  More  specifically,  ISMAT  has  the  following 
capabilities: 

♦  Allocate  functions  and  tasks  to  humans  and  technologies; 

♦  Identify  additional  training  requirements  resulting  from  the  introduction  of  new 
technologies;  and 

♦  Predict  relative  costs. 

ISMAT  incorporates  task  characteristics  (including  task  timelines)  and  operator  knowledge,  skills 
and  abilities  (KSAs)  into  a  dynamic  human  performance  simulation  framework.  Consequences  of 
different  task  allocations  can  be  observed  and  quantitatively  studied  in  the  form  of  cost  analyses, 
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crew  size  requirements  specification,  and  operator  workload  reports.  Of  great  interest  to  this 
project,  ISMAT  supports  the  examination  of  operator  skill  sets  that  are  required  to  perform 
specific  tasks.  Based  on  such  analysis,  operator  knowledge  and  skill  gaps  can  be  identified. 

In  particular,  ISMAT  contains  an  internal  library  that  consists  of  specialized  job-related 
information  (e.g.,  skills  and  proficiency  levels)  from  the  US  Navy’s  personnel  inventory.  A 
generic  human  skill  taxonomy  is  included  in  ISMAT.  This  taxonomy  is  based  on  [3]  and  it 
consists  of  fifty  different  skills  and  abilities  grouped  into  eight  categories,  as  shown  in  Figure  1 1 . 
This  taxonomy  was  adopted  because  its  scales  are  anchored  with  behavioral  examples. 


Job  Assessment  Software  System  -  50  Skills  &  Abilities 


C  omm  unicat  ion 


ORAL  COMPREHENSION 
WRITT EN  C 0M P REH EN SION 
ORAL  EXPRESSION 
WRITTEN  EKPRESSION 


Visual 

NEAR  VISION 
FAR  VISION 
NIGHT  VISION 


Sensory 


VISUAL  COLOR  DISCRIMINATION 
PERIPHERAL  VISION 
DEPTH  PERCEPTION 


C  onceptual 

MEMORIZATION 
PROBLBul  SENSITIVITY 
ORIGINALITY 
FLUENCY  OF  IDEAS 
FLEXIBILITY  OF  CLOSURE 
SELECTIVE  ATT  ENTION 
SPATIAL  ORIENTATION 

visualization  Speed  Loaded 


Perception 


TIMESHARING  S P EED  0 F  CLOS UR E 

PERC  EPTUAL  S  P  EED  CH  0 1C  E  REACTIO  N  TIME 
REACTION  TME 


© 


Cognition 


Reasoning 

INDUCTIVE  REASONING 
CATEGORY  FLEXIBILITY 
DEDUCTIVE  REASONING 
INFORMATION  ORDERING 
MATH  Bui  ATICAL  REASONING 
NUMBER  FACILITY 


GLARE  SENSITIVITY 

Auditory 

GENERAL  HEARING 
AUDITORY  ATTENTION 
SOUND  LOCALIZATION 


Fine  Motor 


CONTROL  PRECISION 
RATE  CONTROL 
WRIST-FINGER  SPEED 
FINGER  DEXTERITY 
MANUAL  DEXTERITY 
ARM-HAND  STEADINESS 
MULTI-LIMB  COORDINATION 

Gross  Motor 

EXTENT  FLEXIBILITY 
DYNAMIC  FLEXIBILITY 
SPEED  OF  LM  B  M  OVEMENT 
GROSS  BODY  EQUILIBRIUM 
GROSS  BODY  COORDINATION 
STATIC  STRENGTH 
DYNAMIC  STRENGTH 
TRUNKSTRENGTH 
STAMINA 


s 


Fleishman.  E.  A.  &  Quaintance.M.  K.  (1984).  Taxonomies  of  human  performance:  The  description  of  human  tasks.  Orlando:  Academic  Press. 


Figure  11:  Fleishman  Human  Skill  Taxonomy 


Based  on  the  Micro  Saint  Sharp  simulation  engine,  ISMAT  allows  analysts  to  study  the 
probabilistic  behavior  of  a  simulation  model  for  assessing  the  interaction  between  crewmembers 
and  assigned  functions  or  tasks  under  the  context  of  various  operational  scenarios.  The  common 
parameters  required  to  run  an  ISMAT  model  include  crew  definition,  maintenance  actions,  and 
scenario  definition. 


Crew  definition  is  achieved  by  adding  operators  to  a  model.  Operators  are  created  either 
automatically  by  importing  pre-defined  crew  documents  or  manually  by  selecting  them  from  a 
library. 

Scenario  definition  involves  the  addition  of  specific  functions  and  tasks  that  the  operators 
perform.  ISMAT  functions  are  structurally  equivalent  to  IPME  networks,  however,  significant 
differences  exist  between  them.  Unlike  IPME  networks,  ISMAT  functions  are  scheduled  on  a 
timeline  and  are  not  required  to  be  decomposed  into  tasks.  Before  executing  an  ISMAT  model, 
the  analyst  must  specify  one  of  three  goals  (i.e.,  minimize  cost,  minimize  workload,  or  minimize 
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crew  size)  for  guiding  model  behavior.  During  model  execution,  1SMAT  attempts  to  allocate 
functions  and  tasks  to  operators  that  best  meet  the  selected  goal. 


Overall,  ISMAT  is  a  flexible  modeling  platform  which  allows  analysts  to  use  simulation  to 
address  personnel  related  ‘what-if  questions.  A  typical  application  of  an  ISMAT  model  includes 
assessing  the  impact  of  crew  reduction  and  alternative  task-operator  assignments. 
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4  Validation  Objectives  and  Methodology 


4.1  Objectives 

With  access  to  occupational  data,  analysts  can  represent  Canadian  Air  Force  personnel  more 
precisely  with  the  addition  of  knowledge  and  skill  attributes  in  a  crew  model.  It  is  now  possible  to 
create  IPME  models  that  are  sensitive  to  these  occupational  characteristics,  and  allow  analysts  to 
investigate  research  questions  related  to  manning  requirements.  Possible  research  questions 
include: 

1 .  What  kind  of  skills  and  knowledge  are  required  by  an  operator  (or  a  group  of  operators) 
to  perform  a  designated  job? 

2.  How  will  changes  of  operator  occupations  affect  system/task  performance? 

3.  What  kind  of  training  is  required  for  an  operator  to  satisfy  job  requirements? 

4.  How  should  skills  and  knowledge  be  prioritized  based  on  their  criticality  in  support  of  job 
performance  at  a  prescribed  level? 

The  objective  of  this  study  was  to  verify  and  validate  the  newly  developed  capability  in  IPME 
and  demonstrate  the  utilization  of  occupational  data  in  human  performance  models  to  address 
such  research  questions. 

4.2  Methodology 

The  generic  human  performance  modeling  process  is  well  established.  With  the  addition  of 
occupational  data  into  modeling  tools,  it  is  possible  to  use  these  data  for  addressing  manning  and 
training  related  questions.  From  a  modeler’s  perspective,  one  critical  modeling  issue  is  the 
association  of  occupational  traits  with  operator  job  performance.  In  IPME,  a  possible  solution  is 
to  use  performance  shaping  functions  (PSFs)  to  create  a  performance  degradation  (or 
improvement)  formula  in  which  the  impact  of  occupational  traits  on  performance  can  be 
specified.  However,  this  requires  sufficient  empirical  evidence  to  justify  the  validity  of  the  PSFs. 
Pragmatically,  evidence  of  this  kind  is  difficult  to  obtain,  and  existing  data  may  not  generalize  to 
different  task  domains.  Therefore,  we  decided  to  explore  an  alternative  approach  by  adopting  a 
Job  Similarity  Index  (JSI)  to  predict  the  level  of  match  between  operators  with  specific  skill  sets 
and  their  designated  tasks. 

Originally  proposed  by  [2],  the  JSI  is  the  ratio  of  the  number  of  task  and  knowledge  (TK) 
statements  required  by  a  task  and  the  number  of  those  required  statements  satisfied  by  an 
operator.  An  operator  must  meet  two  criteria  to  satisfy  a  TK  requirement:  first,  the  operator’s 
occupational  data  (obtained  from  the  MOS  data)  contains  the  TK  statement  required  by  the 
designated  task;  second,  this  operator’s  proficiency  level  for  this  TK  exceeds  the  level  required  by 
the  task.  A  JSI  score  is  calculated  by  dividing  the  number  of  satisfied  TK  statements  by  the  total 
number  of  TK  statements  required  by  this  task.  For  example,  if  an  identification  task  requires 
fifteen  distinct  TKs  and  an  air  navigator  operator  satisfies  ten  of  them,  then  the  air  navigator’s  JSI 
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for  this  task  is  computed  as  10/15  =  0.66.  A  JSI  of  1  indicates  that  an  occupation  fully  satisfies 
the  task’s  requirements,  although  the  operator  may  possess  additional  “unused”  TKs,  which 
reflects  a  case  of  overqualification. 

Farrell  et  al.  hypothesized  that  “a  person  who  has  a  high  JSI  will  perform  well  in  a  new  job  with 
minimal  training,  since  they  already  perform  most  of  the  tasks  and  have  most  of  the  knowledge 
required  for  the  new  job”[2].  This  rationale  was  used  in  our  study  and  JSI  scores  were  computed 
for  the  IPME  model.  However,  since  ISMAT  has  no  internal  constructs  for  representing  TK 
statements,  JSIs  were  not  collected  for  the  ISMAT  model.  Other  conventional  ISMAT  metrics, 
such  as  task  completion  times  and  operator  level  of  effort  were  used  as  performance  indicators. 

In  this  study,  UAV  operation  models  were  created.  The  objective  was  to  identify,  among  existing 
Air  Force  occupations,  the  best  occupations  to  be  assigned  to  three  operator  roles.  Since  we  did 
not  have  a  complete  Air  Force  MOS  database,  the  practical  goal  was  to  examine  the  use  of  MOS 
data  in  simulation  models  for  supporting  such  type  of  analysis. 

A  UAV  operation  scenario,  created  in  a  previous  project  for  investigating  adaptive  and  intelligent 
interface  design  [4],  was  re-used  in  this  study.  The  scenario  is  one  hour  long  and  consists  of  three 
segments,  each  with  a  twenty-minute  duration.  It  begins  with  a  fictitious  advanced  security 
briefing  to  Commonwealth  Heads  of  Government  Meeting  (CHOGM)  security  staff,  conducted  in 
February  201 1.  A  probable  terrorist  threat  is  located  in  a  boat  off  the  coast  of  Newfoundland.  The 
first  segment  is  considered  low  workload  because  the  three-person  UAV  crew  controls  only  a 
single  Vertical  Take-off  UAV  (VTUAV),  which  is  previously  launched  from  a  Canadian  Patrol 
Frigate  (CP- 140),  on  a  regular  surveillance  mission.  Following  initial  system  check-out,  the 
VTUAV  is  required  to  approach  and  record  video  on  one  potential  terrorist  trawler  (designated 
Contact  2),  which  at  the  time  is  conducting  normal  fishing  activities. 

In  the  second  segment,  the  crew  workload  increases.  The  CP- 140  crew  starts  a  second  UAV  (a 
Mini  UAV)  which  is  also  managed  by  the  three-person  UAV  crew.  The  Mini  UAV  monitors  a 
trawler  that  is  fishing  illegally.  At  approximately  the  same  time,  the  VTUAV  is  tasked  to  search 
for  a  new  target,  the  location  of  which  is  initially  not  certain.  The  UAV  crew  has  to  identify  this 
target  among  three  boats.  At  the  request  of  the  UAV  crew,  a  third  UAV  is  deployed.  Together 
with  the  VTUAV,  the  crew  uses  two  UAVs  to  image  these  boats.  At  the  end  of  the  second 
segment,  only  the  third  boat  is  yet  to  be  investigated. 

The  last  segment  begins  with  the  crew  losing  contact  with  the  VTUAV.  In  response,  the  CP- 140 
launches  three  Mini  UAVs  to  monitor  the  third  boat.  At  the  same  time,  two  CF-18s  are  tasked  to 
close  on  the  area.  Boat  #3  is  identified  as  a  terrorist  vessel.  It  launches  a  Lethal  UAV  which  is 
armed  with  a  “dirty”  bomb  and  is  flying  towards  St.  John’s.  The  UAV  crew  needs  to  decide 
proper  counter-measures.  The  scenario  concludes  at  60  minutes  as  the  CP- 140  leaves  the  area  to 
search  for  the  Lethal  UAV  that  has  been  launched.  For  the  UAV  crew,  the  last  segment  represents 
a  high  workload  scenario  due  to  multiple  UAVs  they  are  monitoring  and  time-sensitive  decisions 
that  are  made. 

Annex  A  describes  the  scenario  in  more  detail.  For  complete  information,  readers  are  refered  to 

[4]- 
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This  contrived  UAV  mission  was  modeled  in  both  IPME  and  ISMAT.  The  research  question  was 
to  identify  adequate  occupational  requirements  for  the  UAV  crew.  In  this  case,  the  crew  consists 
of  three  operators:  a  mission  commander  (MC),  a  payload  operator  (PO),  and  a  vehicle  operator 
(VO).  These  operators  were  represented  in  both  IPME  and  ISMAT  models.  Various  occupational 
traits  were  assigned  to  the  operators  and  model  outputs  were  collected  to  identify  the  best- 
qualified  occupation  to  fulfill  each  of  these  three  roles. 

Due  to  a  lack  of  direct  access  to  subject  matter  experts,  there  were  not  sufficient  data  to  create  a 
UAV  model  from  scratch.  A  decision  was  made  to  leverage  other  UAV-related  projects, 
specifically,  a  UAV  operator  selection  study  [2]  and  multiple  UAV  performance  models  [4].  Data 
presented  in  these  two  projects  were  re-used  in  the  current  study  for  constructing  IPME  and 
ISMAT  models.  In  addition,  since  access  to  the  complete  Canadian  Air  Force  occupational  data 
was  not  available  at  the  time  the  study  was  conducted,  only  selected  occupations  from  the 
existing  MOS  database  were  examined  in  the  IPME  model.  Similarly,  because  ISMAT  contained 
only  US  Naval  occupational  data,  the  model  could  only  examine  UAV  operator  selections  from 
those  naval  occupations.  Due  to  these  limitations,  outputs  from  IPME  and  ISMAT  models  could 
not  be  compared  directly.  However,  a  contrast  between  two  modeling  processes,  especially  their 
different  approaches  on  applying  occupational  data,  provides  insights  on  future  modeling  to 
support  personnel  and  crewing  issues. 
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5  IPME  Model 


This  section  describes  the  IPME  UAV  model  that  was  created  for  this  validation  study.  Section 
5.1  explains  the  process  used  to  create  the  model.  Section  5.2  describes  the  model  itself. 

5.1  Modeling  Process 

5.1.1  Available  Data  Sources 

The  data  for  constructing  this  UAV  model  came  from  the  following  sources. 

1.  CFEC  Project  Report  on  the  Alternative  Crew  Selection  Method  [2] 

2.  Hierarchical  Goal  Analysis  and  Performance  Modeling  for  the  Control  of  Multiple 
UAVs/UCAVs  from  an  Airborne  Platform  [4] 

3.  CMC’ s  IPME  models 

4.  MOS  databases 

5.  CFEC  project  database 

Since  most  of  these  data  were  generated  in  former  projects  and  for  different  research  purposes,  it 
was  a  challenge  to  integrate  and  re-use  them  in  a  single  model.  To  construct  a  UAV  model,  we 
extracted  pieces  of  information  from  these  sources  as  required  by  IPME.  Table  7  displays  the  type 
of  information  that  was  obtained  from  each  source  and  the  corresponding  parameters  in  the  IPME 
model. 

Note  that  the  process  involved  in  populating  this  UAV  model  is  not  typical  in  IPME  modeling.  It 
represents  a  work-around  solution  in  response  to  the  lack  of  access  to  subject  matter  experts 
which  was  encountered  in  this  study. 

Table  1:  Sources  for  IPME  UA  V  Model 


Model  Input 


CFEC  Project 
Report  on  the 
Alternative 
Crew  Selection 
Method 


Hierarchical  Goal 
Analysis  and 
Performance  Modeling 
for  the  Control  of 
Multiple  UAVs/UCAVs 
from  an  Airborne 
Platform 


MOS 

Databases 


CFEC 

Project 

Database 


CMC 

IPME 

Models 
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Standard 

Deviation 


Task  Sequence 


Task,  Skill  and 
Knowledge 
Statements  for 
Goals 


Goal  ID 


Task,  Skill  and 
Knowledge 
Statements  for 
Operators 


5.1.2  Determining  Relevant  Tasks  and  Task  Sequence 

Once  data  sources  were  determined,  the  next  step  was  to  identify  the  relevant  mission  scenario 
and,  accordingly,  the  task  sequence  that  the  UAV  crew  needs  to  perform. 

Three  actions  were  taken:  firstly,  execute  existing  IPME  UAV  models  produced  by  CMC 
Electronics  (these  models  are  described  in  [4]  and  referred  as  the  CMC’s  models  in  the  rest  of  the 
report);  secondly,  convert  CMC’s  models  to  a  newer  IPME  format;  and  thirdly,  map  the  task 
identifiers  used  in  the  CMC  models  to  goals  from  the  CFEC  Project  Report  [2], 

Since  the  task  networks  included  in  CMC’s  models  were  based  on  a  specific  mission  scenario 
adopted  in  that  project,  they  could  not  be  reused  in  their  original  form  for  this  study.  However, 
they  did  provide  useful  information  for  determining  the  task  order.  In  this  study,  we  first  reviewed 
the  goal  hierarchy  recorded  in  the  CFEC  Project  Report  [2]  and  then  derived  UAV  operator’s  task 
sequence  based  on  task  order  information  in  CMC’s  models. 

Three  CMC  models  were  originally  created  in  IPME  version  3  and  stored  as  DAT  files.  This  file 
format  is  no  longer  supported  by  the  latest  version  of  IPME  (i.e.,  version  4),  so  the  files  had  to  be 
converted  to  a  newer  format  to  facilitate  analysis.  A  three-step  translation  scheme  was  followed. 

1 .  the  DAT  files  were  loaded  into  IPME  version  3  and  then  exported  in  an  XML  file  format 
(i.e.,  SYX).  The  SYX  data  format  is  supported  by  the  latest  version  of  IPME  (i.e.,  v4); 

2.  when  the  SYX  files  were  loaded  into  IPME  v4,  the  model  expressions  were  automatically 
translated  into  the  newer  syntax  used  in  IPME  version  4; 

3.  after  the  conversion  was  completed  for  all  three  DAT  files,  they  were  aggregated  into  a 
single  project  file  (i.e.,  XML  file  format  that  contains  multiple  systems). 

This  process  is  illustrated  in  Figure  12. 
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Figure  12:  UAV  Model  Data  Files  Conversion  Process 

In  CMC’s  original  models,  the  operational  sequence  diagram  (OSD)  identifiers  were  used  to  label 
tasks.  Since  these  OSD  IDs  were  numerical  numbers,  it  caused  difficultly  in  interpreting  task 
sequences.  To  address  this  issue,  a  four-step  process  was  taken  to  convert  the  OSD  ID-based  task 
names  into  meaningful  textual  descriptors,  and  also  to  identify  tasks  that  could  be  re-used  in  the 
new  model.  This  process  is  illustrated  in 

Figure  13. 


1.  Starting  with  the  Goal  IDs  obtained  from  [2],  we  located  the  corresponding  OSD  IDs  in 
the  Hierarchical  Goal  Analysis  (HGA)  output  contained  in  Annex  C  of  [4]. 

2.  We  then  searched  the  project  file  that  contained  three  CMC  models  for  task  names  with 
identical  OSD  IDs.  This  produced  a  lsit  of  task  IDs. 

3.  The  same  OSD  ID  was  then  located  in  the  OSD  diagrams  (from  the  Annex  D  of  [4])  to 
identify  the  corresponding  Goal  ID. 


Figure  13:  Steps  Taken  to  Match  Goal  IDs,  OSD  IDs,  and  Task  IDs 
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4.  The  OSD  diagrams  listed  a  node  name  and  a  Goal  ID  for  each  OSD  ID.  When  the  Goal 
ID  in  the  OSD  diagram  matched  the  Goal  ID  obtained  from  [2]  (i.e.,  step  1),  the  IPME 
task  ID  was  recorded  as  the  corresponding  task  ID  for  that  goal  ID.  These  tasks  were 
regarded  as  tasks-of-interest  for  this  study,  and  were  re-used  to  construct  the  new  UAV 
model. 

After  all  re-useable  IPME  tasks  were  identified,  the  CMC  models  were  run  in  IPME  (version 
3.0.26)  to  collect  its  execution  trace  files.  The  re-useable  tasks  were  then  located  in  the  trace  files. 
Based  on  their  order  in  the  trace  files,  new  task  sequences  were  then  obtained  for  constructing  the 
new  UAV  model  for  this  project. 

Basic  task  parameters  for  the  new  model,  such  as  task  mean  times  and  standard  deviations,  were 
directly  re-used  from  the  CMC  models.  [4]  provides  such  information  for  most  of  the  modeled 
tasks. 

Table  8  shows  a  portion  of  the  result  from  this  mapping  process.  A  complete  conversion  table  is 
available  in  Annex  B. 


Table  8:  Illustration  of  the  Mapping  between  Goal  IDs,  OSD  IDs,  and  Task  IDs 


Goal  ID  and 
Goal 

Descriptor 

Completion 
Time  (sec) 

OSD 

ID 

OSD 

Operator 

CMC 

Scenario 

Segment 

OSD 

Page 

Number 

CMC 

IPME 

Model 

Task  ID 

2.1.1  radar 

4-9 

164 

TN 

2 

2 

21.12.11 

plot  of  an  area 

166 

TN 

2 

2 

21.12.12 

of  interest  is 
current 

167 

UP 

3 

7 

27.6.5 

3.5.2  a 

determination 
of  the  SAC 
(OSC)  is 
completed 

12 

196 

TN 

3 

10 

27.12.3 

3.5.1 

command  & 
control  of 
tactical 
situation  is 
assessed 

12 

192 

TN 

3 

10 

27.12.1 

5.1.3  Identifying  Task  and  Knowledge  (TK)  Statements  based  on  Task 
Goals 

After  a  new  task  network  for  the  UAV  model  was  completed,  the  next  step  was  to  generate  a  list 
of  task  and  knowledge  statements  for  each  task.  These  statements  represented  the  TKs  required 
by  these  tasks  and  were  derived  based  on  the  goals  associated  with  each  task. 
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The  following  list  of  data  were  produced  by  this  process: 

1.  A  list  of  duty  areas,  including  their  names  and  codes; 

2.  A  list  of  TK  statements  associated  with  each  goal  ID;  and 

3.  A  unique  list  of  the  names  of  each  TK  statement  used  in  the  model. 

Once  we  had  a  set  of  goals,  we  proceeded  to  identify  the  TK  statements  for  each  goal.  After 
obtaining  permission  to  use  a  CFEC  project  database  (i.e.,  Data  source  5),  a  query  was  performed 
to  retrieve  a  complete  list  of  the  TK  statements  required  for  each  goal.  Note  that  some  of  the  goals 
included  in  this  query  were  not  included  in  the  IPME  4  UAV  model  because  there  were  no 
corresponding  tasks  in  the  CMC  IPME  models.  The  goal  data  for  goals  not  included  were 
ignored. 

Next,  character  replacements  were  performed  on  the  list  of  TK  statements  to  ensure  the 
statements  conformed  to  IPME  naming  restrictions.  The  following  replacements  were  made  in  the 
listed  order. 

1.  Replace  all  characters  except  &,  /,  letters  (a-z,  A-Z),  and  digits  (0-9)  with  a  space 

(e.g-,“”). 

2.  Replace  &  with  the  string  “  and  ”  (note  the  whitespace  surrounding  the  word). 

3.  Replace  /  and  spaces  (“  ”)  with  underscores  (e.g.,  “_”). 

4.  Replace  any  sequence  of  two  or  more  underscores  with  a  single  underscore. 

A  list  of  unique  statements  was  generated  using  the  sort  and  filter  features  of  Microsoft  Excel. 
Minor  capitalization  corrections  were  made  to  the  resulting  list.  This  process  produced  a  list  of 
unique  TK  statements. 

The  CFEC  database  categorized  each  statement  into  a  duty  area  identified  by  a  single  letter  label 
(i.e.,  A-Z).  However,  duty  areas  in  CAMA  were  identified  by  a  textual  descriptor.  A  translation  of 
the  duty  area  letter  labels  to  the  textual  descriptions  was  required.  Each  task  statement  was 
located  in  the  CAMA  databases  and  all  matching  duty  areas  were  retrieved  (a  single  statement 
can  belong  to  multiple  duty  areas).  The  duty  area  descriptor  that  had  the  most  occurrences  was 
assumed  to  be  the  duty  area  name  which  corresponded  to  the  letter  of  the  task  statement.  The 
result  of  this  matching  process  is  shown  in  Table  9. 

For  duty  areas  N,  S,  V,  Y,  and  Z,  no  matches  were  found  in  any  of  the  C  AMA  databases,  so  no 
duty  area  name  suggestion  could  be  made. 

5.1.4  Phase  1:  Evaluate  Performance  of  Candidate  Operators 

The  first  step  in  building  an  IPME  model  was  to  define  the  UAV  operator  tasks.  Such  tasks  were 
based  on  information  obtained  from  the  experiment  described  in  the  CFEC  Project  Report  [2]. 
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These  tasks  were  performed  by  three  UAV  operators:  a  mission  commander  (MC),  a  payload 
operator  (PO),  and  a  vehicle  operator  (VO). 


Table  9:  Duty  Area  Labels  and  Names 


Duty  Area  Label  Duty  Area  Name 

A 

Air  Traffic  Control 

B 

Air  Defence 

C 

Navigation 

D 

Maritime  Operations 

E 

Air  Duties 

F 

Search  and  Rescue  Operations 

G 

Air  Missile  Warning  Space  Operations 

H 

Common  Operations 

I 

General  Aircrew  Activities 

J 

Mission  Flight  Planning 

K 

Electronic  Warfare 

L 

Communications 

M 

Maintenance  Duties 

N 

N/A 

0 

Tactical  Level  HQ  Operations 

P 

Operational  Strategic  Level  HQ  Operations 

Q 

Mission  Support  Activities 

R 

Surveillance 

S 

N/A 

T 

Policy  Framework  and  Organisational  Structures 

U 

Information  Management 

V 

N/A 

w 

General  Military  Requirements 

Y 

N/A 

Z 

N/A 

Currently  there  are  no  specific  Air  Force  occupations  defined  to  serve  these  roles.  The  CFEC 
study  [2]  identified  tasks  requirements  for  each  of  these  roles  based  on  an  analysis  of  their 
operational  requirements.  This  information  was  re-used  to  populate  the  IPME  task  network. 

The  next  step  was  to  define  three  operators  (i.e.,  MC,  PO,  and  VO)  in  the  IPME  crew  model. 
Traits  (shown  in  Figure  14)  were  added  to  each  operator  that  represented  the  knowledge  and  skills 
each  operator  possessed  to  perform  the  predicted  MC,  VO,  and  PO  job  elements.  After  the  crew 
model  was  completed,  each  operator  task  in  the  task  network  model  was  statically  assigned  to  one 
operator  based  on  information  in  [2].  By  using  a  static  operator  assignment,  the  assumption  was 
that  the  selected  operator  would  always  perform  the  designated  task.  After  the  operators  were 
defined  and  populated,  the  crew  model  was  saved  to  the  master  database  for  re-use  in  Phase  2. 


30 


DRDC  CR  2009-059 


£3  Project  Tree 


?  £3  MOSID 

2S  MOSID 
o-  »S  NetModell 

BestPossibleCrew 
v  &  MC 

o-  9  Anthropometry 
»•  9  DefauItVars 
6-  9  Properties 
9  States 
f  9  Traits 

*-  l%  MOSID_Air_Defence 
«-  MOSID_Air_Duties 

o-  ^  MOSID_Air_Missile_Warning_Space_Operations 
«-  3s  MOSID_Air_Mobility_Operatlons 
®-  MOSID_Air_Trafflc_Control 

o-  i%  MOSID_Common_Operations 
o-  3$  MOSID_Communications 
o-  %  MOSID_Electronic_Warfare 
o-  ^  MOSID_Financial_Management 
o-  %  MOSID_Flight_Safety 
o-  %  MOS!D_General_Aircrew_Activities 
o-  i$)  MOSID_General_Leadership_Command_Discipline 
o-  &  MOSID_Oeneral_Leadership_Command_and_Discipline 
o-  ^  MOSID_General_Mllitaiy_Requirements 
o-  MOSID_Ground_Duties 

o-  ,%  MOSID_HQ_Operations 
o-  %  MOSIDJnformation_Management 
o-  ^  MOSID_Maintenance_Duties 
o-  St  MOSID_Maritime_Operations 
o-  Sj  MOSID_Mission_Flight_Planning 
o-  3$  MOSID_Mission_Support_Activities 
o-  %  MOSID_Navigation 
o-  MOSID_On_A_C_General_Aircrew_Activities 

o-  3&  MOSID_Operational_Strategic_Level_HQ_Operations 
o-  3&  MOSID_Personnel_Management_and_Administration 
o-  %  MOSID_Policy_Framework_and_Organizational_Structures 
o-  MOSID_Project_Contract_Management 

o-  ^  MOSID_Resource_Management 
o-  3$  MOSID_Search_and_Rescue_Operations 
o-  ^  MOSID_Standards_and_Training 
 o-  ^  MOSID Tactical Level HQ Operations 


Figure  14:  Operator  Traits 


Next,  a  candidate  occupation,  such  as  “Air  Navigator,”  was  specified  for  each  of  the  emerging 
jobs.  The  associated  task  and  skill  proficiency  levels  were  then  populated  in  the  operator  model. 
Logic  was  added  in  each  IPME  task  to  calculate  the  Job  Similarity  Index  (JSI)  for  the  assigned 
operator.  For  example,  a  JSI  could  be  calculated  for  the  “Air  Navigator”  performing  a  task  in  the 
role  of  a  mission  commander  (MC).  The  TK  statements  possessed  by  the  operator  were  compared 
to  the  TK  statement  required  by  a  particular  task.  A  ‘match’  was  confirmed  when  the  operator 
possessed  the  required  TK  statement  and  the  operator’s  proficiency  level  met  or  exceeded  the 
task’s  minimum  proficiency  requirement.  If  no  minimum  proficiency  level  was  specified  for  the 
task’s  TK  statement,  any  operator  proficiency  level  for  that  TK  statement  was  considered  a 
match.  The  total  number  of  “matches”  was  divided  by  the  total  number  of  TK  statements  required 
by  the  task  to  determine  the  JSI  for  the  operator- task  pair. 


A  default  minimum  proficiency  level  of  3  was  initially  defined  for  all  TK  statements.  This  value 
was  selected  because  it  is  near  the  average  of  all  proficiency  values  in  the  database.  This  default 
value  can  be  changed  manually  in  the  modeling  process. 
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5.1.5  Phase  2:  Identify  Additional  Candidate  Occupations 


After  the  task  network,  crew  model,  and  data  collection  snapshots  were  created,  alternative  crew 
options  were  examined  by  substituting  crew  models  containing  alternate  candidate  occupations. 
These  crew  options  were  represented  by  crew  models  inserted  into  the  IPME  system  using 
IPME’s  plug-and-play  capability.  The  process  involved  the  following  steps. 

1.  An  alternative  candidate  occupation  was  identified  for  each  of  three  modeled  operators. 

2.  A  new,  unassigned  crew  model  was  added  to  the  IPME  project.  This  crew  model  was 
populated  by  linking  to  the  operators  saved  to  the  master  database  in  Phase  1 . 

3.  The  modeler  repeated  the  process  described  in  Phase  1  for  the  second  set  of  candidate 
occupations. 

4.  Using  the  plug-and-play  capability  of  IPME,  the  task  network  model  was  executed  again 
with  a  different  crew  model. 

The  development  process  in  both  phases  was  conducted  iteratively.  The  model  was  tested  and 
executed  repeatedly,  with  the  intent  to  provide  a  model  that  could  be  re-used  and  minimize  the 
risk  associated  with  obtaining  data  from  external  sources. 

5.2  IPME  Model 

The  top-level  task  network  of  the  UAV  model,  shown  in  Figure  15,  contained  three  sub-networks 
(“65  Model  1”,  “66  Model2”,  and  “67  Model3”),  each  representing  a  distinct  CMC  model,  a  data 
collection  task  (“73  Snapshot  Task”),  a  task  that  parses  the  text  files  used  to  populate  necessary 
arrays  (“72  ParseTextFiles”),  and  some  continuous  tasks  called  by  multiple  sub-networks  (“68 
73.33  CONT  Mini  UAV”  and  “69  7.3.5.10  CONT  EO”). 


Figure  15:  Top  Level  Network 

In  each  network,  there  are  smaller  groups  of  tasks  connected  by  paths.  The  first  task  in  each  group 
is  given  a  name  similar  to  “4. 1 .14.”  This  number  refers  to  the  matching  sub-network  in  the 
corresponding  CMC  model.  For  example,  all  tasks  that  follow  “4.1.14”  were  in  the  4.1.14  sub¬ 
network  in  the  CMC  model.  The  first  task  in  each  group  is  not  assigned  to  an  operator  and  does 
not  take  any  time  (i.e.,  task  mean  time  equals  zero).  All  other  tasks  are  given  names  that  indicate 
their  goal  id,  e.g.,  “10.1.2.1  MC  workstation  is  configured.”  These  tasks  are  assigned  to  operators 
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and  consume  time.  The  last  task  in  a  task  sequence  calls  a  function  that  determines  which  task 
sequence  to  start  next.  The  purpose  of  using  such  a  function  for  connecting  task  branches  was  to 
improve  the  readability  of  the  task  network  model  and  provide  an  easy  approach  for  modifying 
the  task  execution  order. 


B> 


K  kj66-29414  KG694  2.1.1  '^W-kJ66'621'1  ralla,]/Vkl66'1325'2'1  bs. 

^T>0.  ^  Jv^Hvtuav  radar  is  specific  |\/^plot  of  an  area  Js/^plot  of  an  area  Js/^location  of  all  J\/ 
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^66.16121.17  |0>>^VTUAV  raIlar  |S  j^^yTUAV  flgtt  f/  ^VTUAV  Jv^yrUAV  route  to  JO~^ynjAV  heading  )Q^*(loeation  of  all  jQ^VTUAV  route  is  f\/j 

H 66.18  9.4.1 .4  1/s  .  166.19  9.4.1.4  Vs^92994299  ]/\»f66.21  9.4.1 .4  l/\vj66.23  9.4.1 ,4  Un 
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[66.103  7.1 .3 
UAV  flight 
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Figure  16:  Sub-network  that  Represents  Goals 


This  model  reads  in  data  from  three  CSV  files: 


1.  “TaskIDstoGoallDsSorted.csv”  contains  a  mapping  of  IPME  task  IDs  to  goal  IDs. 

2.  “UniqueStatementList.csv”  contains  the  names  of  each  task  and  knowledge  statement  that 
is  used  in  the  model. 

3.  “GoallDStatementDutyAreaList.csv”  contains  the  list  of  task  and  knowledge  statements 
required  by  each  goal  ID. 

Four  snapshots  were  added  to  collect  simulation  outputs  using  the  plain  text  format  option  in 
IPME  version  4.  The  snapshots  are  described  in  detail  in  Section  5.2.3. 


5.2.1  Crew  Model 

It  took  some  thoughtful  consideration  to  determine  the  best  method  for  adding  the  TK  statements 
to  the  crew  model.  Manually  searching  through  each  of  the  four  MOS  databases  would  have 
required  2,284  key  word  searches  to  locate  the  571  unique  TK  statements  using  CAMA.  This  was 
considered  unrealistic  and  an  alternative  solution  was  sought. 

Instead,  the  MOS  databases  were  searched  using  the  same  database  query  that  identified  the  duty 
area  name.  This  query  showed  that: 

1.  354  statements  did  not  exist  in  any  of  the  CAMA  databases, 
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2.  159  statements  had  a  unique  match  in  one  of  the  CAMA  databases,  and 

3.  58  statements  matched  multiple  duty  areas  in  one  or  more  of  the  C  AMA  databases. 

To  resolve  the  58  ambiguous  statements,  the  statement  belonging  to  the  assumed  duty  area  name 
was  selected  as  the  best  match.  If  the  duty  area  for  the  statement  did  not  match  the  assumed  duty 
area  name,  then  this  occurrence  was  not  considered  a  match.  A  matching  statement  belonging  to 
the  correct  duty  area  can  occur  multiple  times  in  the  databases  if  it  is  associated  with  multiple 
jobs.  Since  each  occurrence  can  have  a  different  proficiency  value,  the  proficiency  value  from  the 
first  matching  occurrence  was  always  used. 

After  disregarding  TK  statements  that  were  not  available  in  CAMA,  a  total  of  2 1 7  TK  statements 
remained.  These  TK  statements  were  added  to  the  crew  model.  Note  that  this  implementation  was 
not  efficient  because  it  resulted  in  unused  statements  being  added  to  the  crew  model  as  operator 
traits.  Minor  usability  and  efficiency  issues  resulted  since  it  took  analysts  more  time  to  locate  the 
TK  statement  in  the  long  trait  lists  and  the  saved  model  file  was  larger.  However,  since  these 
unused  traits  were  not  accessed  in  the  task  network  model,  they  had  no  impact  on  the  model 
outputs,  and,  therefore  did  not  affect  the  validity  of  the  model. 

By  assigning  the  operators  in  this  crew  model  every  TK  statement  available  from  the  MOS 
databases,  we  represented  the  best  possible  operator.  This  provided  us  with  the  most  desirable 
JSI  values  which  could  be  used  as  a  benchmark  for  more  realistic  operator  structures.  Note  due  to 
the  missing  354  TK  statements  in  the  CAMA  databases,  the  JSI  computation  for  the  best  possible 
operator  would  not  generate  a  perfect  score  (i.e.,  1). 

5.2.2  Alternative  Crew  Model 

Two  sets  of  candidate  occupations  were  compared  to  assess  how  operators  from  existing  AF 
occupations  would  perform  in  the  MC,  PO,  and  VO  roles.  The  selected  candidate  occupations 
were  chosen  based  on  the  completeness  of  information  in  the  MOS  databases.  The  modeller 
possessed  no  information  that  suggested  either  of  the  two  candidate  occupations  were  strong 
candidates  for  the  MC,  PO,  and  VO  positions. 

1.  Three  operators  were  assumed  to  possess  occupational  training  as  air  mobility  navigators. 
That  is,  the  operators  possessed  the  skills  specified  in  the  TK  statements  for  an  air 
mobility  navigator.  For  documentation  purpose,  these  TK  data,  together  with  the 
proficiency  scores,  are  currently  stored  in  the  MOSID-ANAV  database.  The  resulting 
crew  model  is  named  “ANAV182AirNavl62AirMobilInstrNav.” 

2.  In  the  other  crew  model,  each  of  the  three  operators  possessed  occupational  training  as  a 
maritime  helicopter  tactical  control  officer.  That  is,  the  operators  each  had  acquired  the 
task  and  knowledge  statements  for  a  maritime  helicopter  tactical  control  officer.  This  TK 
data,  together  with  the  proficiency  scores,  were  stored  in  MOSID-PILOT  database.  The 
resulting  crew  model  is  named  “PILOT182AirNav203MarHeliTacCO.” 

These  two  crew  models  were  assigned  to  the  IPME  UAV  system  model  in  place  of  the  original 
“best  possible  operator”  crew  model.  The  impact  of  the  different  crew  compositions  was  reflected 
in  the  JSI  outputs. 
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5.2.3  Model  Output 

Since  the  default  IPME  outputs  were  not  sufficient  for  this  study,  snapshots  were  created  to 
collect  customized  simulation  data.  Snapshots  are  internal  constructs  in  IPME  for  recording  the 
values  of  selected  variables  at  specified  points  during  model  execution.  These  data  can  be  used  to 
generate  statistics  and  graphs.  For  the  EJAV  IPME  model,  four  snapshots  were  created. 

0  JSISnap  -  This  snapshot  recorded  the  name  of  the  operator  assigned  to  this  task,  the  JSI  value 
for  the  assigned  operator,  and  the  task  name  (which  includes  the  goal  ID). 

0  JSIValues  -  This  snapshot  recorded  the  name  of  the  operator  assigned  to  this  task,  the  JSI 
value  for  the  assigned  operator,  and  the  name  of  the  task  (which  includes  the  goal  id).  This 
snapshot  was  easily  imported  into  Microsoft  Excel  for  further  analysis, 
o  NoGoalFound  -  This  snapshot  recorded  the  goal  IDs  for  which  we  have  task  and  knowledge 
statements,  but  were  not  included  in  the  IPME  model. 

0  TasksNoRequirements  -  This  snapshot  recorded  the  goal  IDs  for  tasks  which  do  not  have  any 
required  task  or  knowledge  statements. 

5.3  Results 

5.3.1  Crew  Results 

Table  10-Table  12  list  the  results  of  the  “JSIValues”  snapshot.  Note  that  the  snapshots  “JSISnap” 
and  “JSIValues”  contain  identical  data;  only  their  format  is  different.  JSI  scores  for  the  “best 
possible  crew”  did  not  equal  1.0  due  to  a  practical  constraint  of  this  study  (missing  354  TK 
statements  in  the  MOS  databases).  A  cell  value  of  “N/A”  indicates  that  the  task  was  not 
performed,  so  the  JSI  value  was  not  calculated. 

Table  10:  MC  Operator  JSI  Values  per  Task 


Task  ID  Task  Name  Best  Possible  Air  Maritime 

Crew  JSI  Mobility  Helicopter 

Instructor  Tactical 
Navigator  Control 
Crew  JSI  Officer 
Crew  JSI 

2.1.1 

radar  plot  of  an  area  of  interest  is  current 

0.36 

0.21 

0.31 

2.5.2. 1 

location  of  all  unknown  unit  icons  on  tactical  plot 

0.36 

0.18 

0.32 

3.5.1 

command  and  control  of  tactical  situation  is  assessed 

0.41 

0.07 

0.41 

3.5.2 

determination  of  the  SAC  (OSC)  is  completed 

0.34 

0.07 

0.34 

4.4.10 

contact  can  be  identified 

0.00 

0.00 

0.00 

5.1.3 

relevant  rules  of  engagement  are  reviewed 

0.26 

0.04 

0.26 

5.2.1 

overt  actions  of  terrorist  unit  personnel  are  observed 

0.17 

0.05 

0.18 

5.3.1 

potential  weapons  onboard  terrorist  unit 

0.24 

0.11 

0.24 

5.3.8 

risk  to  neutral  units 

0.13 

0.13 

0.13 

5.5.2 

selection  of  best  units  to  counter  terrorist  threat 

0.29 

0.11 

0.29 
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Task  ID  Task  Name  Best  Possible  Air  Maritime 

Crew  JSI  Mobility  Helicopter 

Instructor  Tactical 
Navigator  Control 
Crew  JSI  Officer 
Crew  JSI 

5.5.2 

selection  of  best  units  to  counter  to  terrorist  threat 

0.29 

0.11 

0.29 

5.5.6 

selection  of  the  best  offensive  systems 

0.29 

0.11 

0.29 

5.6.3 

tasking  message  has  been  transmitted 

0.26 

0.14 

0.25 

5.7.8 

damage  report  provided  to  friendly  unit 

0.31 

0.19 

0.31 

6.4.1 

the  probable  position  of  the  terrorist  vessel 

0.27 

0.16 

0.25 

6.4.4 

search  area  is  appropriate  for  current  situation 

0.20 

0.10 

0.20 

6.4.5 

display  area  on  workstation  is  appropriate 

0.33 

0.22 

0.33 

6.8.1 

need  for  contingency  plans  is  addressed 

1.00 

1.00 

1.00 

6.8.2 

contingency  plans  are  created 

0.41 

0.41 

0.41 

6.8.3 

contingency  plans  are  discussed 

0.00 

0.00 

0.00 

7.1. 1.8 

VTUAV  activities  are  planned 

0.00 

0.00 

0.00 

7.1.1.11 

handover  of  VTUAV  has  been  prepared 

0.26 

0.26 

0.26 

7.3. 1.2 

Mini  UAV  search  pattern  is  planned 

0.38 

0.31 

0.38 

7.3.1.10 

crew  is  briefed  on  use  of  Mini  UAVs 

0.43 

0.43 

0.43 

7.3.1.11 

crew  have  been  requested  to  launch  Mini  UAV 

0.00 

0.00 

0.00 

133.5 

Mini  UAV  is  autonomously  following  contact 

0.00 

0.00 

0.00 

7.3.5.10 

CONT  EO  images  are  organized  on  desktop 

1.00 

0.00 

0.00 

7.3.8. 1 

previous  minutes  of  video  are  reviewed 

0.25 

0.00 

0.25 

93.23 

fighter  aircraft  are  directed  to  attack  threat 

0.26 

0.16 

0.24 

9.4. 1.5 

information  of  a  general  nature 

0.22 

0.00 

0.11 

10.1.2.1 

MC  workstation  is  configured 

0.25 

0.00 

0.25 

2.1.1 

radar  plot  of  an  area  of  interest  is  current 

0.36 

0.21 

0.31 

2.5.2. 1 

location  of  all  unknown  unit  icons  on  tactical  plot 

0.36 

0.18 

0.32 

3.5.1 

command  and  control  of  tactical  situation  is  assessed 

0.41 

0.07 

0.41 

Table  11:  PO  Operator  JSI  Values  per  Task 


T ask  ID  Task  N ame 

Best  Possible  Crew  JSI  Air 

Maritime 

Mobility 

Helicopter 

Instructor 

Tactical 

Navigator 

Control 

Crew  JSI 

Officer 
Crew  JSI 

36 


DRDC  CR  2009-059 


Task  ID  Task  Name  Best  Possible  Crew  JSI  Air  Maritime 

Mobility  Helicopter 
Instructor  Tactical 
Navigator  Control 
Crew  JSI  Officer 
Crew  JSI 

2.1.4 

tactical  plot  icons  are  current 

0.32 

0.18 

0.28 

4.1.5 

latest  position  of  all  unknown  contacts  is  plotted 

0.35 

0.21 

0.30 

4.2.5 

contact  is  sought  using  UAV  radar 

0.17 

0.09 

0.17 

4.4.5 

crew  identifying  vessel  using  UAV  EO  suite 

0.29 

0.06 

0.29 

4.4.7 

UAV  images  of  boat  are  compared  with  database 

0.20 

0.07 

0.20 

4.4.8 

crew  classify  vessel  using  UAV  EO  suite 

0.60 

0.00 

0.60 

5.6.4 

tasking  message  has  been  acknowledged 

0.26 

0.16 

0.24 

7. 1.4.1 

VTUAV  EO  sensor  settings  are  optimized 

0.21 

0.00 

0.21 

6.9.1 

VTUAV  piloting  aspects  are  studied 

N/A 

N/A 

0.38 

7.1. 1.1 

VTUAV  route  to  the  next  operating  area  is  planned 

N/A 

N/A 

0.41 

7. 1.4.2 

VTUAV  EO  sensor  is  used  for  a  test  observation 

1.00 

0.00 

1.00 

7.1.1.10 

VTUAV  route  is  plotted 

N/A 

N/A 

0.50 

7.1. 1.2.1 

selection  of  an  appropriate  search  pattern 

N/A 

N/A 

0.37 

7. 1.5. 7 

VTUAV  EO  image  fde  is  stowed 

0.33 

0.00 

0.33 

7. 1.9.1 

VTUAV  data  uplink  is  maintained 

0.20 

0.00 

0.20 

7.1. 1.2.7 

location  of  contact  symbol  is  determined  on  tacplot 

N/A 

N/A 

0.43 

7. 1.9. 3 

initial  system  settings  on  VTUAV  are  reviewed 

0.10 

0.10 

0.10 

7.1. 1.6.1 

location  of  potential  VTUAV  refueling  platforms 

N/A 

N/A 

0.45 

7.3.4. 1 

Mini  UAV  EO  sensor  settings  are  optimized 

0.14 

0.00 

0.14 

7.1. 1.6.2 

estimated  CPF  location  at  time  off  task 

N/A 

N/A 

0.38 

13.5.2 

Mini  UAV  EO  sensor  is  used  to  study  a  contact 

0.25 

0.00 

0.25 

7.1. 1.6.4 

rough  VTUAV  time  on  task  is  calculated 

N/A 

N/A 

0.00 

7. 3. 5.4 

Mini  UAV  EO  zoomed  in  on  a  portion  of  boat 

0.00 

0.00 

0.00 

7.1. 1.6.6 

any  problems  associated  with  refueling 

N/A 

N/A 

0.00 

13.5.6 

Mini  UAV  EO  sensor  is  used  to  track  a  contact 

0.14 

0.05 

0.14 

7. 1.2.1 

VTUAV  heading  has  changed  to  a  new  heading 

N/A 

N/A 

0.36 

13.5.1 

Mini  UAV  EO  is  used  to  record  high  definition  images 

0.00 

0.00 

0.00 

7. 1.2.1 

CONT  VTUAV  heading  has  changed  to  a  new  heading 

N/A 

N/A 

0.36 

7. 3. 5. 8 

Mini  UAV  FOV  trapezoid  is  over  contact 

0.00 

0.00 

0.00 

7. 1.2.2 

VTUAV  altitude  has  changed  to  a  new  altitude 

N/A 

N/A 

0.34 

9.4. 1.3 

identification  and  activities  of  contact 

0.24 

0.14 

0.19 

7. 1.2.4 

VTUAV  autopilot  set  to  autonomous  mode 

N/A 

N/A 

0.17 
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Task  ID  Task  Name  Best  Possible  Crew  JSI  Air  Maritime 

Mobility  Helicopter 
Instructor  Tactical 
Navigator  Control 
Crew  JSI  Officer 
Crew  JSI 

7.1.3 

VTUAV  flight  path  is  monitored 

N/A 

N/A 

0.36 

10.1.2.3 

PO  workstation  is  configured 

0.25 

0.25 

0.25 

7. 1.5. 6 

VTUAV  EO  images  are  monitored 

N/A 

N/A 

0.33 

7. 1.5. 7 

VTUAV  EO  image  file  is  stowed 

N/A 

N/A 

0.33 

7. 1.9.2 

initial  system  checks  on  VTUAV  are  conducted 

N/A 

N/A 

0.25 

7. 1.9.4 

VTUAV  systems  are  monitored 

N/A 

N/A 

0.00 

7.3. 1.8 

Mini  UAV  route  is  plotted 

N/A 

N/A 

0.33 

7.3.2. 1 

Mini  UAV  heading  has  changed  to  a  new  heading 

N/A 

N/A 

0.29 

1.222 

Mini  UAV  altitude  has  changed  to  a  new  altitude 

N/A 

N/A 

0.31 

7.3.3. 1 

Mini  UAV  symbol  has  appeared  on  the  surface  plot 

N/A 

N/A 

1.00 

1322 

Mini  UAV  is  in  descent  following  deployment 

N/A 

N/A 

0.33 

1333 

CONT  Mini  UAV  is  following  the  planned  flight  path 

N/A 

N/A 

0.50 

13.12 

initial  system  checks  on  Mini  UAV  are  conducted 

N/A 

N/A 

0.44 

13.13 

Mini  UAV  systems  are  monitored 

N/A 

N/A 

0.33 

13.1  A 

Mini  UAV  systems  are  managed 

N/A 

N/A 

0.33 

9.4. 1.4 

specific  information  regarding  a  UAV 

N/A 

N/A 

0.00 

Table  12:  VO  Operator  JSI  Values  per  Task 


Task  ID  Task  Name  Best  Possible  Air  Mobility  Maritime 

Crew  JSI  Instructor  Helicopter 

Navigator  Tactical 

Crew  JSI  Control 

Officer  Crew 
JSI 

6.9.1 

VTUAV  piloting  aspects  are 
studied 

0.41 

0.36 

0.38 

7. 1.9.2 

initial  system  checks  on  VTUAV 
are  conducted 

0.30 

0.20 

0.25 

7.1. 1.1 

VTUAV  route  to  the  next 
operating  area  is  planned 

0.44 

0.41 

0.41 

7.1.3 

CONT  VTUAV  flight  path  is 
monitored 

0.36 

0.32 

0.36 

7.1.1.10 

VTUAV  route  is  plotted 

0.50 

0.50 

0.50 
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Task  ID  Task  Name  Best  Possible  Air  Mobility  Maritime 

Crew  JSI  Instructor  Helicopter 

Navigator  Tactical 

Crew  JSI  Control 

Officer  Crew 
JSI 

7.1. 1.2.1 

selection  of  an  appropriate  search 
pattern 

0.37 

0.29 

0.37 

7.1. 1.6.4 

rough  VTUAV  time  on  task  is 
calculated 

0.00 

0.00 

0.00 

7.1. 1.2.7 

location  of  contact  symbol  is 
determined  on  tacplot 

0.43 

0.14 

0.43 

7.1. 1.6.1 

location  of  potential  VTUAV 
refueling  platforms 

0.45 

0.29 

0.45 

7.1. 1.6.2 

estimated  CPF  location  at  time  off 
task 

0.38 

0.13 

0.38 

7. 1.2.1 

CONT  VTUAV  heading  has 
changed  to  a  new  heading 

0.40 

0.36 

0.36 

7.1. 1.6.6 

any  problems  associated  with 
refueling 

0.00 

0.00 

0.00 

9.4.1.4 

specific  information  regarding  a 
UAV 

0.00 

0.00 

0.00 

7. 1.2.2 

VTUAV  altitude  has  changed  to  a 
new  altitude 

0.41 

0.34 

0.34 

7. 1.2.4 

VTUAV  autopilot  set  to 
autonomous  mode 

0.17 

0.17 

0.17 

7.1.3 

VTUAV  flight  path  is  monitored 

0.36 

0.32 

N/A 

7. 1.9.4 

VTUAV  systems  are  monitored 

0.00 

0.00 

0.00 

7. 1.5.6 

VTUAV  EO  images  are 
monitored 

0.33 

0.00 

0.33 

7. 1.9.2 

initial  system  checks  on  VTUAV 
are  conducted 

0.30 

0.20 

0.25 

7.3.3. 1 

Mini  UAV  symbol  has  appeared 
on  the  surface  plot 

1.00 

0.00 

1.00 

733.2 

Mini  UAV  is  in  descent  following 
deployment 

0.33 

0.00 

0.33 

13.1.2 

initial  system  checks  on  Mini 

UAV  are  conducted 

0.56 

N/A 

0.44 

7.3. 1.8 

Mini  UAV  route  is  plotted 

0.33 

0.33 

0.33 

7.3.2. 1 

Mini  UAV  heading  has  changed 
to  a  new  heading 

0.43 

0.29 

0.29 

13.2.2 

Mini  UAV  altitude  has  changed  to 
a  new  altitude 

0.38 

0.31 

0.31 

13.13 

Mini  UAV  systems  are  monitored 

0.33 

0.33 

0.33 

13.1  A 

Mini  UAV  systems  are  managed 

0.33 

0.33 

0.33 
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Task  ID 

Task  Name 

Best  Possible 
Crew  JSI 

Air  Mobility 

Instructor 

Navigator 

Crew  JSI 

Maritime 
Helicopter 
Tactical 
Control 
Officer  Crew 
JSI 

7.3.33 

CONT  Mini  UAV  is  following 
the  planned  flight  path 

0.50 

0.25 

0.50 

Table  13  lists  the  minimum,  maximum,  and  average  JSI  values  for  the  three  operators.  The  tasks 
where  the  JSI  was  equal  to  1.0  are  listed,  which  indicates  a  perfect  match  of  the  operator’s  TK 
statements  to  that  task’s  TK  requirements.  Upon  further  analysis,  it  would  have  been  beneficial  to 
record  any  tasks  with  zero  JSI  scores,  since  such  scores  indicate  that  the  assigned  operator  does 
not  possess  any  relevant  knowledge  or  skills,  indicating  a  complete  qualification  gap. 

Table  13:  Summary  of  Operator  JSI  Values 


Crew  and  Operator  Average  Minimum  Maximum  Tasks  with  JSI= 1.0 


JSI  JSI  JSI 


Best  Possible  Crew 

MC 

0.29 

0 

1 

6.8.1  need  for 
contingency  plans  is 
addressed 

7.3.5.10  CONT  EO 
images  are  organized  on 
desktop 

VO 

0.35 

0 

1 

7.3.3. 1  Mini  UAV  symbol 
has  appeared  on  the 
surface  plot 

PO 

0.25 

0 

1 

7. 1.4.2  VTUAV  EO 
sensor  is  used  for  a  test 
observation 

Air  Mobility 

Instructor  Navigator 
Crew 

MC 

0.15 

0 

1 

6.8.1  need  for 
contingency  plans  is 
addressed 

VO 

0.33 

0 

1 

7.3.3. 1  Mini  UAV  symbol 
has  appeared  on  the 
surface  plot 

PO 

0.23 

0 

0.4140256 

None 

Maritime  Helicopter 
Tactical  Control 

Officer  Crew 

MC 

0.25 

0 

1 

6.8.1  need  for 
contingency  plans  is 
addressed 

VO 

0.33 

0 

1 

7.3.3. 1  Mini  UAV  symbol 
has  appeared  on  the 
surface  plot 

PO 

0.24 

0 

1 

7. 1.4.2  VTUAV  EO 
sensor  is  used  for  a  test 
observation 
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5.3.2  Discussion 


Ideally,  we  would  continue  to  explore  substituting  additional  occupations  into  the  crew  model  to 
achieve  the  highest  JSI  possible.  Due  to  time  constraints  of  the  project,  only  two  such 
substitutions  were  performed.  The  average  JSI  results  of  the  crews  are  presented  in  Table  13.  A 
comparison  of  the  JSI  results  for  the  Air  Mobility  Navigator  and  the  Maritime  Helicopter  Tactical 
Control  Officer  is  presented  in  Table  14. 


Table  14:  JSI  Scores  as  a  Percentage  of  Best  Possible  Crew  JSI 


Operator 

Air  Mobility  Navigator 

Maritime  Helicopter 
Tactical  Control 
Officer 

MC 

50.5% 

86.5% 

VO 

18.7% 

94.2% 

PO 

91.7% 

96.3% 

The  following  basic  conclusions  were  drawn  from  the  simulation  results. 

1.  None  of  the  selected  occupations,  including  the  hypothetical  ‘best  possible  crew,’  was 
able  to  satisfy  all  of  the  knowledge  and  skills  requirements  imposed  by  three  UAV 
operator  positions. 

2.  Overall,  the  JSI  scores  were  fairly  low.  However,  if  you  compare  the  JSI  scores  to  the 
best  possible  crew  JSI  scores,  the  best  match  is  obtained  by  placing  the  Maritime 
Helicopter  Tactical  Control  Officer  in  the  PO  role.  Placing  the  Air  Mobility  Navigator  in 
the  VO  role  resulted  in  the  worst  match. 

3.  The  Maritime  Helicopter  Tactical  Control  Officer  was  “more  qualified”  to  serve  in  all 
three  roles  than  the  Air  Mobility  Instructor  Navigator,  as  shown  by  higher  JSI  scores. 


The  objective  of  this  modeling  exercise  was  to  identify  CF  occupations  that  satisfied  the 
knowledge  and  skills  requirements  imposed  by  each  of  three  UAV  operator  roles.  An  ideal 
occupation  (if  it  exists)  would  generate  a  JSI  score  of  one  for  all  designated  tasks.  Any  imperfect 
JSI  result  reflects  a  potential  opportunity  for  performance  breakdown  and  is  an  indication  that 
further  training  is  needed.  Since  currently  there  are  no  Air  Force  occupations  that  correspond 
directly  to  each  of  these  UAV  operator  jobs,  perfect  JSI  scores  could  not  be  generated.  The  ‘best 
possible  crew’  in  the  study  should  be  inteipreted  as  the  ‘best  possible’  solution  available  from  the 
existing  MOS  database.  Without  a  complete  occupational  dataset  from  the  Canadian  Air  Force, 
the  conclusion  was  not  informative.  However,  it  did  demonstrate  the  usefulness  of  incorporating 
such  a  hypothetical  crew  model  for  assessing  the  amount  deviation  of  an  actual  occupation  (e.g.,  a 
maritime  helicopter  tactical  control  officer)  in  fulfilling  job  requirements  from  that  of  a 
hypothetical  “best  possible’  solution. 

The  JSI  scores  presented  in  Section  5.3.1  provide  some  insight  about  the  goodness  of  fit  between 
selected  AF  occupations  and  the  designated  UAV  operator  roles.  An  occupation  with  a  higher  JSI 
score  is  interpreted  as  a  better  fit  to  these  roles.  However,  it  is  important  to  note  that  the 
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quantitative  JSI  scores  can  not  be  used  directly  for  interpreting  operator  performance.  For 
example,  for  the  VO  role,  the  Maritime  Flelicopter  Tactical  Control  Officer  generated  a  JSI  5.5 
times  larger  than  that  of  an  Air  Mobility  Navigator.  However,  this  does  not  imply  that  the 
Maritime  Helicopter  Tactical  Control  Officer  would  perform  5.5  times  better.  Implications  of  this 
nature  need  to  be  studied  in  the  conventional  framework  of  IPME  modeling,  using  performance 
shaping  functions  that  define  the  numerical  relations  between  operator  characteristics  and 
performance  indicators  (e.g.,  task  execution  times  or  system  failure  rates).  Such  methods  could 
also  be  used  to  identify  minimum  acceptable  JSI  scores  for  each  task.  With  the  support  of 
additional  data,  the  relationship  between  occupational  definitions  and  task  performance  could  be 
studied  in  a  more  robust  IPME  model. 
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6  ISMAT  Model 


6.1  ISMAT  Modeling  Process 

The  Integrated  Simulation  Manpower  Analysis  Tool  (ISMAT)  was  specifically  designed  and 
developed  to  support  U.S.  Navy  ship  manning  concepts  and  was  not  intended  to  be  a  general 
purpose  human  performance  modeling  tool.  As  such,  ISMAT  includes  a  large  library  of  U.S. 
Navy  crew,  organization,  and  equipment  data.  There  is  a  large  maintenance  manpower  analysis 
portion  that  was  not  utilized  for  this  effort.  The  scope  of  the  functional  analyses  that  can  be 
modeled  with  ISMAT  includes  operations,  facilities  maintenance,  planned  preventive 
maintenance,  and  unplanned  corrective  maintenance.  Only  the  operations  portion  was  used  in 
this  study. 

ISMAT  integrates  skill  and  ability  requirements  with  the  dynamic  human  performance  simulation 
framework.  ISMAT  allows  the  analyst  to  examine  the  skills  required  to  perform  specific  tasks 
and  can  also  describe  operators  in  terms  of  what  skills  they  possess  at  what  level.  ISMAT 
contains  a  library  that  provides  information  on  specialized  skills  and  proficiency  levels  that  are 
available  within  the  US  Navy’s  enlisted  personnel  inventory,  classified  by  rate  and  rating. 

In  ISMAT,  the  user  describes  each  task  in  terms  of  the  human  abilities  required  to  perform  it. 
Using  ISMAT,  an  analyst  first  systematically  proceeds  through  the  set  of  50  skills  and  abilities 
and  decides  which  ones  are  required  for  performing  a  particular  task.  The  next  step  is  to 
determine  the  demand  level  for  the  task.  Each  skill  is  rated  on  a  scale  from  0  to  70.  The  user  can 
either  manually  enter  a  score  or  use  a  sliding  scale  to  pinpoint  the  proper  location. 

Once  the  user  has  determined  what  KSAs  are  required  for  a  task,  ISMAT  queries  the  crew 
profiles  and  presents  the  user  with  a  list  of  all  existing  operators  that  meet  or  exceed  the  task  KSA 
requirements.  The  users  can  then  select  one  or  more  operators  from  the  list,  or  define  new 
operators  manually.  The  process  of  defining  a  new  operator  involves  entering  a  name,  defining 
the  rank,  rating,  and  the  associated  skills  and  abilities. 

While  the  generic  ISMAT  modeling  approach  is  available  in  its  users’  manual  [8],  the  following 
list  representing  a  simplified  view  of  the  modeling  process  is  provided  for  the  reader’s 
convenience. 

1 .  Crew  parameter  definition 

This  is  where  the  characteristics  of  crew  are  defined.  This  process  consists  of  two  parts:  firstly  the 
selection  of  crew  members  and  secondly  the  specification  of  such  crew  parameters  as  skills, 
availability  (work  hours  per  week)  and  cost. 

2.  Scenario  and  function  parameters  definition 

ISMAT  functions  are  analogous  to  IPME  networks.  The  definition  of  functions  involves  the 
specification  of  task  parameters  such  as  skills  and  abilities  requirements,  whereas  the  definition  of 
scenarios  is  accomplished  by  adding  specific  functions  and  tasks  to  operators.  The  top  level 
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networks  are  defined  using  a  GANTT-Chart  style  editor.  The  top-level  functions/networks  can 
optionally  be  decomposed  to  the  task  level. 

3.  Assign  crew  to  functions/tasks 

Each  non-automated  function  or  task  must  be  assigned  to  one  or  more  operators.  1SMAT 
searches  the  crew  skill  profiles  and  shows  the  user  which  crew  member(s)  meet  or  exceed  the 
skill  levels  required  by  the  function/task.  If  the  crew  member(s)  do  not  meet  or  exceed  the  skills 
requirements  they  are  highlighted  in  red.  The  user  specifies  the  required  crew  size  for  each 
function  /  task  as  well  as  which  operators  might  perform  the  function/task.  This  allows  user  to 
optionally  develop  operator  "pools"  that  allow  dynamic  allocation  of  jobs  to  tasks  to  occur  as  the 
simulation  is  running.  Whenever  a  task  is  scheduled  to  begin,  ISMAT  determines  the  “best” 
operator(s)  according  to  the  user  specified  goal  (e.g.,  minimize  cost,  minimize  crew  size, 
minimize  workload)  and  operator  availability. 

4.  Model  executition 

Once  the  analysis  has  been  defined,  ISMAT  automatically  generates  a  discrete-event  simulation 
model  from  the  user’s  inputs  and  executes  the  tasks  as  they  would  be  performed  in  a  real 
scenario,  taking  into  consideration  the  variability  in  task  performance  time  and  accuracy.  Unlike 
a  static  flow  diagram  of  activities,  the  models  created  with  ISMAT  are  able  to  simulate  the 
interplay  of  activities  and  influences  between  crewmembers,  automation  devices,  and  conditions 
outside  of  the  environment  of  the  system  being  modeled. 

5.  Result  analysis 

ISMAT  automatically  generates  several  reports  that  summarize  operator  skill  usage,  function  or 
task  performance,  and  the  level  of  operator  utilisation.  Such  results  are  applied  for  diagnosing 
system  designs.  Users  can  also  define  their  own  data  collection  snapshots. 

6.2  UAV  ISMAT  Modeling  Approach 

The  version  of  ISMAT  used  in  this  study  was  version  1.1,  distributed  on  27  July  2006.  This 
section  provides  a  detailed  account  of  the  ISMAT  model  development  process,  particularly  the 
challenges  that  were  encountered  and  their  corresponding  solutions. 

1.  UAV  model  conversion 

In  order  to  save  time  and  reduce  the  possibility  of  errors,  we  used  several  different  ways  of 
automatically  importing  the  model  data  directly  from  IPME. 

The  initial  import  attempt  used  data  from  IPME’s  Spreadsheet  Editor.  The  relevant  model  data, 
e.g.,  the  name  and  meantime  columns  in  the  Spreadsheet  Editor,  were  first  copied  and  pasted  to  a 
Microsoft  Excel  spreadsheet.  Additional  manual  editing  was  required  to  produce  a  tab-delimited 
text  file  compatible  with  ISMAT.  However,  because  this  method  was  time-consuming  and 
difficult  to  reproduce  (e.g.,  in  case  of  IPME  model  changes),  a  temporary  solution  was  used  by 
adding  a  new  menu  option  “Save  Network  as  ISMAT  File”  in  a  development  version  of  IPME. 
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This  option  simplified  the  process  of  importing  task  information,  such  as  names  and  mean  times, 
but  other  type  of  data  still  had  to  be  manually  entered. 


Alternatively,  we  explored  the  use  of  a  Microsoft  Excel  data  import  format  supported  by  ISMAT. 
This  option  allowed  the  automatic  convertion  of  intra-task  pathways  into  an  ISMAT  format. 

Since  this  feature  was  revealed  to  be  desirable  for  general  modeling  purpose,  a  new  export 
function  was  added  to  the  IPME  in  which  a  new  menu  option  “Save  as  CSV”  was  added  to  the 
task  network  and  crew  models  in  the  commercially  distributed  version  of  IPME.  Note  the  CSV 
file  generated  by  this  menu  command  does  not  record  the  complete  model  data,  only  those  that 
are  used  for  the  ISMAT  model  is  captured  in  the  CSV  file.  Once  the  CSV  files  for  the  task 
network  and  crew  models  were  generated,  they  were  manually  converted  to  the  Microsoft  Excel 
format.  The  list  below  shows  the  relevant  task  information  supported  by  this  import  process. 


•  Task  Name 

•  Task  ID 

•  Mean  Time 

•  Time  Distribution 

•  Standard  Deviation 

•  Beginning  Effect 

•  Ending  Effect 

•  Following  Task(s) 


Due  to  syntax  differences  between  IPME  and  ISMAT,  minor  changes  were  made  to  the  model 
after  it  was  imported  into  ISMAT.  In  particular,  any  continuous  task  in  IPME  was  manually 
converted  into  a  series  of  three  tasks  in  ISMAT  (see  Figure  17). 


KsTARTCONTTASK69(1_1_136)$' — K  69  7.3.5.10  CONTATTEND  EO  images  are  organized  on  desktop  (1_1_133)  ^->^CONTTASK69  NON  ATTEND  (1_1_134) 

Figure  17:  Continuous  Task  Group  in  ISMAT 

In  IPME,  a  continuous  task  can  be  defined  which  has  two  time  parameters:  an  attending  time  and 
a  non-attending  time  [1].  Such  construct  is  not  available  in  ISMAT,  as  a  result,  tasks  in  ISMAT 
are  always  discrete.  In  order  to  translate  an  IPME  continuous  task  into  ISMAT,  a  series  of  three 
discrete  tasks  was  used  in  ISMAT  (see  Figure  17).  Within  this  task  group,  the  first  task  is  a 
starter  and  it  contains  logics  that  compute  the  active  time  (the  mean  time  for  the  attending  task), 
the  total  duty  cycle  time,  and  the  mean  time  for  the  non  attending  task;  the  following  two  tasks 
represent  the  attending  and  non-attending  portion  of  the  continuous  task  respectively.  In  this 
study,  the  starter  task  was  not  assigned  to  any  crew  member,  while  the  attending  and  non 
attending  tasks  were  assigned  to  the  associated  operator. 

For  some  tasks  in  the  IPME  model,  their  standard  deviation  for  task  completition  time  used 
internal  variables  such  as  Entity.MeanTime.  Such  expressions  could  not  be  used  in  an  ISMAT 
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model.  Consequently,  adjustments  were  made  and  those  expressions  were  changed  to  their 
numeric  equivalent  in  the  1SMAT  model. 

In  addition,  the  IPME  model  contained  three  nextTask()  functions  that  started  tasks  in  each  of  the 
sub-networks  via  start()  function  calls.  These  nextTask()  functions  were  removed  in  the  ISMAT 
model  and  replaced  with  paths  connecting  the  tasks.  This  modification  did  not  change  the  order  in 
which  tasks  executed. 

2.  Crew  definition 

Crew  selection  in  the  ISMAT  model  presented  a  series  of  challenges.  ISMAT  contains  a  library 
of  jobs  selected  from  the  US  naval  occupations.  Because  of  the  lack  of  direct  mapping  between 
MOS  TK  statements  (used  in  IPME)  and  the  Fleishman  human  skill  taxonomy  (used  in  ISMAT), 
the  IPME  crew  could  not  be  directly  imported  into  ISMAT.  In  consultation  with  the  project’s 
scientific  authority,  it  was  decided  to  select  similar  (i.e.,  not  identical)  occupations  for  defining 
crew  members  in  the  ISMAT  model. 

A  first  attempt  was  made  which  included  a  two-step  process  for  identifying  naval  occupations  in 
ISMAT  that  were  similar  to  the  Canadian  Air  Force  occupations  tested  in  the  IPME  model.  First, 
we  identified  US  Air  Force  occupations  that  corresponded  to  the  Canadian  ones;  second,  we 
mapped  those  US  AF  occupations  to  the  US  Navy  occupations.  The  basis  for  the  mapping  was 
knowledge  and  skills  requirements. 

Specifically,  operators’  job  titles  that  were  tested  in  IPME  (e.g.,  an  air  mobility  instructor 
navigator  or  a  maritime  helicopter  tactical  control  officer),  were  used  as  keywords  for  searching 
the  Occupational  Information  Network  (0*NET)  which  is  available  through 
http://online.onetcenter.org.  The  0*NET  database  includes  occupational  information  for  the 
entire  US  military.  The  following  steps  were  taken  to  identify  candidate  occupations  to  use  in  the 
ISMAT  model 

1 .  Search  for  these  keyword  strings  in  the  0*NET  database. 

2.  Scan  the  list  of  search  results  for  related  US  AF  occupations. 

3.  Find  the  corresponding  0*NET  occupations. 

4.  Find  the  corresponding  US  Navy  occupations. 


Table  15  and  Table  16  display  results  obtained  for  mapping  to  the  air  mobility  instructor 
navigator  and  the  maritime  helicopter  tactical  control  officer  respectively. 
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Table  15:  Candidate  Occupations  related  to  an  Air  Mobility  Instructor  Navigator 


US  AF  US  Navy  0*NET 

81  TOW  Instructor,  General 
(Air  Force  -  Commissioned 
Officer  only) 

8235  Instructor,  General 

25-3099.99  Teachers  and 
Instructors 

12M1Y  Mobility  Navigator, 
General  (Air  Force  - 
Commissioned  Officer  only) 

No  match  found. 

53-201 1.00  Airline  Pilots, 
Copilots,  and  Flight 
Engineers 

12M1Z  Mobility  Navigator, 
Other  (Air  Force  - 
Commissioned  Officer  only) 

No  match  found. 

53-201 1.00  Airline  Pilots, 
Copilots,  and  Flight 
Engineers 

Table  16:  Candidate  Occupations  related  to  a  Maritime  Helicopter  Tactical  Control  Officer 


US  AF  US  Navy  0*NET 

1 1H1 Y  Helicopter  Pilot, 
General  (Air  Force  - 
Commissioned  Officer  only) 

No  match  found. 

53-2012.00  Commercial 
Pilots 

1 1H1Z  Helicopter  Pilot, 

Other  (Air  Force  - 
Commissioned  Officer  only) 

No  match  found. 

53-2012.00  Commercial 
Pilots 

13B1  Command  and  Control 
Operations  (Air  Force  - 
Commissioned  Officer  only) 

Closest  match:  0342  Global 
Command  and  Control  System 
Common  Operational 
Picture/Maritime  (GCCS 
COP/M)  Operator 

55-1015.00  Command  and 
Control  Center  Officers 

7380  Tactical  System 
Officer/Mission  Specialist 
(Marine  Corps  -  Warrant 
Officer  only) 

No  match  found. 

53-201 1.00  Airline  Pilots, 
Copilots,  and  Flight 
Engineers 

0322  LAMPS  MK  III  Air 
Tactical  Control  Operator 
(Navy  -  Enlisted) 

55-3015.00  Command  and 
Control  Center  Specialists 

Additionally,  queries  were  conduced  in  the  Military  Occupational  Classification  (MOC)  to 
Standard  Occupational  Classification  (SOC)  Crosswalk  database.  This  database  supports  queries 
using  occupation  codes  from  different  classification  systems,  e.g.,  dictionary  of  occupational 
titles,  military  occupational  classification,  standard  occupational  classification,  registered 
apprenticeship  information  system,  and  classification  of  instructional  programs.  The  outputs  of 
these  queries  were  three  US  naval  occupations: 

•  8235  Instructor,  General 

•  0342  Global  Command  and  Control  System  Common  Operational  Picture/Maritime 
(GCCS  COP/M)  Operator 
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•  0322  LAMPS  MK  III  Air  Tactical  Control  Operator  (Navy  -  Enlisted) 

However,  an  internal  search  in  ISMAT  manning  documents  revealed  that  none  of  these 
occupations  were  available  in  existing  ISMAT  databases.  The  reason  none  of  the  occupations 
existed  is  because  the  codes  obtained  from  0*NET  are  not  the  same  type  of  codes  used  in 
ISMAT.  The  analyst  should  have  been  looking  for  the  Primary  Rating  field  in  0*NET  rather 
than  the  MOC  field  in  order  to  find  matching  descriptors.  In  ISMAT  version  1.1 1  operators  are 
defined  in  terms  of  rating  (BM  -  boatswain  mate,  OS-  operations  specialist,  etc.)  and  rank  (El- 
recruit,  E2-apprentice,  E3-seaman,  E4-petty  officer  3ld  class,  etc.).  The  MOC  codes  that  were 
obtained  via  the  0*NET  and  crosswalk  search  are  Navy  Enlisted  Codes  (NECs)  or  Navy  Officer 
Billet  Classification  (NOBC)  codes.  In  the  U.S.  Navy  each  sailor  has  one  rank  and  rating  but  may 
have  multiple  NECs  depending  on  what  schools  and  training  they  have  attended.  Therefore,  this 
approach  failed  to  generate  matching  occupations  in  the  ISMAT  database  that  were  representative 
of  the  occupations  tested  separately  in  the  IPME  model. 

An  alternative  attempt  was  made  by  examining  the  occupational  database  for  a  CVN  68  Nimitz- 
class  aircraft  carrier.  This  process  produced  the  following  list  of  occupations  that  were  eventully 
applied  in  the  ISMAT  model,  as  shown  in  Figure  18. 


Selected  Crew 


Billet 

Name 

Rating 

Grade 

Department 

Division 

00001 

ELECTRONICS 

TECHNICIAN 

ET 

E4 

OPERATIONS 

OE 

00002 

RADIOMAN 

RM 

E5 

OPERATIONS 

OC 

00003 

OPERATIONS 

SPECIALIST 

OS 

E7 

OPERATIONS 

Ol 

00004 

ALL-NAVY 

ANYBODY 

E9 

EXECUTIVE 

EXECUTIVE 

00005 

ALL-NAVY 

ANYBODY 

E1_3 

AIR 

V 

00006 

SIGNALMAN 

SM 

E1_3 

NAVIGATION 

N 

00007 

SIGNALMAN 

SM 

E7 

NAVIGATION 

N 

00008 

BOATSWAIN  S  MATE 

BM 

E4 

DECK 

2ND 

00009 

ALL-NAVY 

ANYBODY 

El  3 

ENGINEERING 

A 

Figure  18:  Screenshot  of  Initial  Selected  Crew  for  ISMAT  UAV  Model 


Note  that  different  from  the  IPME  model  where  operators  (with  occupational  specification)  were 
statically  assigned  to  each  task,  ISMAT  was  able  to  automatically  choose  an  operator  for  each 
task  based  on  the  scenario  goal.  In  order  to  take  advantage  of  this  capability  in  ISMAT,  a  pool  of 
virtual  operators  with  a  variety  of  occupational  backgrounds  (see  Figure  16)  was  created  and  used 
in  the  test.  The  entire  operator  pool  was  assigned  to  each  task,  so  that  the  build-in  ISMAT 
algorithm  could  work  effectively  to  determine  the  approproriate  operator  assignment 
dynamically. 

1  Note:  ISMAT  version  2.0  includes  NEC/NOBC  profiles  as  well  as  0*NET  profiles  for  describing  crew 
members. 
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6.3  The  ISMAT  Model 


The  completed  ISMAT  model  consisted  of  three  functions  (networks)  and  397  tasks.  The  critical 
tasks,  as  identified  by  [2]  are  listed  in  Table  17  along  with  their  corresponding  task  IDs  in  the 
ISMAT  model.  A  complete  list  can  be  found  in  [1]. 

Table  1 7:  Critical  Task  Sequence 


Goal  ID 

Description 

ISMAT 

Task  IDs 

4.4.5 

crew  identifying  vessel  using  UAV  EO  suite 

65_35 

65  3  9 

65_47 

65  50 

66  65 

66  67 

4.4.6 

ISAR  imagery  is  downloaded  and  analysed 

676 

4.4.7 

UAV  images  of  boat  are  compared  with  database 

66  54 

4.4.8 

crew  classify  vessel  using  UAV  EO  suite 

66  5  5 

66  5  9 

4.4.9 

legality  of  fishing  boat  activities 

66  68 

66_75 

4.4.10 

contact  can  be  identified 

66_78 

For  each  critical  task,  skill  requirements  were  specified  based  on  the  Fleishman  human  skill 
taxonomy.  Skill  requirements  were  not  defined  for  any  of  the  non-critical  tasks.  Due  to  the  lack  of 
supporting  data,  the  skill  assignments  were  made  arbitrarily  in  this  study.  A  sample  assignment  is 
provided  in  Table  18.  This  was  considered  acceptable  since  the  intent  was  to  examine  the 
modeling  process. 
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Table  18:  Critical  Tasks  Required  Skill  Set 


Skill  Name 

Score 

Deductive  Reasoning 

49 

General  Hearing 

41 

Inductive  Reasoning 

35 

Oral  Comprehension 

35 

Oral  Expression 

37 

A  single  scenario,  named  UAV,  was  created  for  the  model.  This  scenario  had  three  functions: 
coded  65,  66,  and  61  respectively.  They  were  identical  to  three  sub-networks  in  the  IPME  model. 
Each  function  was  decomposed  into  mostly  the  same  tasks  (with  an  exception  on  continous  tasks, 
as  discussed  before)  as  the  IPME  model.  Additionally,  the  IPME  model  had  a  few  dummy  tasks 
for  data  collection;  they  were  removed  in  the  ISMAT  model.  Figure  19  shows  a  scenario 
schedule,  with  each  function  set  to  one  hour  in  duration,  and  Figure  20  illustrates  a  portion  of  the 
decomposed  sub-task  network  for  Function  65. 


>-  ISMAT:  MOSID  UAV  050408.ismat 


0e  Edit  View  Define  Execution  Reports  Utilities 

J9Q  ..  V  *  JJ  « 


Help 


Q- 


Q  Analysis  Properties/7!^  UAV  (1)  | _ 

Scenario:  |  UAV  (1 )  3  j_|  J 


LT 


□  ID  Name 

Day  1 

p 

op  qe  op  ip 

17  1  3  65  Mo  dell 

17  1  2  66  Model2 
17  1  1  67  ModeD 


Figure  19:  UAV  Scenario  Schedule 


^65_97.1.4.1  VTUAVEO  sensor  settings  are  optimized  (1_3_97)$>»{65_107.1. 9.2  initial  system  checks  on  VTUAV  are  conducted  ( 1_3_96)  ^ 

U  65 J 1 6.9.1  VTUAV  piloting  aspects  are  studied  (1_3_95)  65.137  1  42  VTUAV  E0  sensor  is  used  fora  test  observation  (1. 

I§3(K 


C  ►(  START  C  0  NT  T  ASK  65_53  ( 1  _3_1 06)  $>W  65_53CONT  7.1.9.1  VTUAV  data  uplink  is  maintained  (1.3.54)  $>  M  CQNT  TASK  65.53  NON 


*•  1  J  1  65_1 47.1.1.6.3  VTUAV  fuel  on  board  and  average  fuel  flow  (1_3_92)  $>-»{  65J67.1.1.64  rough  VTUAVtir 


65_19  7.1 .1 .6.1  location  of  potential  VTUAV  refueling  platforms  (1_3_87)  65_21 7.1 .1 .6.2  estimated  CPF  location  at  time  off  task  (1_3_8^) 

65_23  7.1 .1.6.5  precise  VTUAV  time  or.  task  is  calculated  (1_3_83)~$1-' »(  65_24  94.1 .2  '.'TUAV  calculated  time  on  task  is  transmitted  i,  1_3_82)j^ 

Figure  20:  Partial  Task  Decomposition  of  Function  65 
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Human  tasks  in  the  ISMAT  model  were  populated  with  timing  information.  In  this  study,  each 
ISMAT  task’s  timing  data  (e.g.,  task  completion  mean  time,  and  standard  deviation)  were 
identical  to  their  counterpart  in  the  IPME  model.  Task  skill  requirements  were  specified  using  the 
Fleishman  skill  taxonomy  for  critical  tasks  only.  For  other  task  parameters,  the  ISMAT  default 
values  were  used.  Error!  Reference  source  not  found,  and  Error!  Reference  source  not 
found,  show  how  a  typical  ISMAT  task  is  populated  with  both  timing  information  and  skill 
requirements. 


Figure  21:  Task  timing  information  in  ISMAT 


File  Edit  Vjew  Define  Execution  Reports  Utilities  [Help 

j  a  Q  ii  JU  •/  *  M  ,  Q* 

Analysis  Properties  [3  UAV  (1)X 0  66_78  4.4.10  contact  can  be  identified  (1_2_63)  | 

•r  <  >  X 

66_78  4.4.10  contact 
can  be  identified 

j  ">|rrs  |  m 

E 

Item  ID:|63 

r 

Timing  Skills /Automation  |  Failuie  Parameters  |  Advanced!  Effects!  Paths  |  Queue  |  Appearance  and  Notes  | 


How  should  this  task  be  performed? 

Personnel  Required  Potential  Exists  for  Automation  Automated 


Personnel  Requirements  |  Automation  Potential  Required  Skills  | 


All  Library  Skills  And  Abilities 


Name 

Description 

SELECTIVE 

ATTENTION 

The  ability  to  concentrate  on  a  task  one 
is  doing.  This  ability  includes 
concentrating  while  performing  a 
boring  task  and  not  being  distracted. 

SPATIAL 

ORIENTATION 

The  ability  to  tell  where  you  are  in 
relation  to  the  location  of  some  object 
or  to  tell  where  the  object  is  in  relation 

VISUALIZATIO 

N 

The  ability  to  imagine  how  something 
will  look  when  it  is  moved  around  or  its 
parts  are  moved  or  rearranged.  It 
requites  the  forming  of  mental  images 

CATEGORY 

FLEXIBILITY 

The  ability  to  produce  many  rules  so 
that  each  rule  tells  how  to  group  a  set 
of  things  in  a  different  way.  Each 
different  group  must  contain  at  least 

INFORMATIO 

N  ORDERING 

The  ability  to  follow  correctly  a  rule  or 
set  of  rules  to  arrange  things  or  actions 
in  a  certain  order.  The  rule  or  set  of 
rules  used  must  be  given. 

MATHEMATIC 

AL 

REASONING 

The  ability  to  understand  and  organize 
a  problem  and  then  select  a 
mathematical  method/formula  to  solve 
the  problem.  It  encompasses 

NUMBER 

FACILITY 

Involves  the  degree  to  which  adding, 
subtracting,  multiplying,  and  dividing 
can  be  done  quickly  and  correctly. 

Skills  Abilities  Required  to  Perform  Function 


Name 


ORAL  COMPREHENSION 
INDUCTIVE  REASONING 
DEDUCTIVE  REASONING 
ORAL  EXPRESSION 
GENERAL  HEARING 


Choose  a  category  of  skills  to  view:  I  Show  All  Skills 


3  I*  Show  Score  ToolTips 


Figure  22:  Task  skills  requirement  in  ISMAT 
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6.3.1  Model  Output 

ISMAT  automatically  produces  twenty  three  different  reports  as  shown  in  Figure  23. 


Reports  1  Utilities  Help 

% 

Manpower  Requirements  -  Operational 

► 

Manpower  Requirements  -  Directed 

► 

Manpower  Requirements  -  Maintenance 

► 

OperatorUtilization 

► 

=3* 

Utilization  Over  Time 

► 

Overall  Skill  Usage 

► 

m 

Skill  Usage  Over  Time 

CrewComposition 

m 

Performance 

► 

Mean  vs.  Actual  Function  Time 

► 

Manpower  Cost 

► 

£ 

Equipment  Cost 

m 

Maintenance  Hit  Matrix 

B 

Maintenance  Execution  Details 

m 

Task  Hit  Matnx 

m 

Function/Task  Execution  Details 

m 

Operator  Activity 

m 

Operator  Conflicts 

Planned  vs.  Actual  Time 

Operator  Level  Of  Effort 

► 

Level  Of  Effort  Over  Time 

► 

m 

Operator  Situational  Awareness 

m 

Task  Failure 

Figure  23:  ISMAT  Reports 

Output  parameters  include  manhour  requirements,  planned  versus  actual  mission  performance,  a 
listing  of  functions/tasks  that  were  delayed  due  to  manpower  constraints,  relative  cost,  equipment 
failure,  maintenance  manhour  requirements,  and  skill  and  ability  usage. 

While  an  ISMAT  model  produces  a  number  of  different  reports,  only  the  Overall  Skill  Usage 
report  was  examined  for  this  project.  This  is  the  report  that  most  closely  matches  the  JSI  Values 
reported  in  the  IPME  model  results  section. 
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•  Overall  Skill  Usage  -  This  report  can  be  used  to  describe  the  skills  that  are  needed  to 
perform  a  particular  job  during  the  conduct  of  the  simulated  scenario.  It  shows  the 
cumulative  skill  usage  by  job  for  all  8  major  skill  categories  (Auditory,  Communication, 
Conceptual,  Fine  Motor,  Gross  Motor,  Reasoning,  Speed  Loaded,  and  Visual).  The  report 
also  shows  the  percentage  of  total  skill  usage  that  can  be  attributed  to  each  skill  category. 

This  report  can  be  viewed  either  in  tabular  or  graphical  format. 

A  sample  output  from  this  UAV  model  (see  Figure  24)  showed  that  the  operators  used  Auditory, 
Communication,  and  Reasoning  skills  when  performing  the  mission  critical  tasks. 


Figure  24:  Overall  Skill  Usage  Report 


In  this  particular  case,  this  result  was  expected  because  only  5  skills  out  of  50  possible  were 
assigned  to  the  critical  tasks  as  shown  in  Table  19. 

Table  19:  Skills  and  Categories  Assigned  to  Critical  Tasks 


Skill  Name 

Category 

Score 

Deductive  Reasoning 

Reasoning 

49 

General  Hearing 

Auditory 

41 

Inductive  Reasoning 

Reasoning 

35 

Oral  Comprehension 

Communication 

35 

Oral  Expression 

Communication 

37 

Overall,  it  is  difficult  to  identify  a  single  occupation  as  the  “best”  to  perform  the  tasks  in  ISMAT 
without  consultation  with  a  SME  to  obtain  missing  data  such  as  operator  level  of  effort,  task 
priority,  and  skill  requirements.  The  model  outputs  indicate  that  it  is  more  appropriate  to  select 
required  skills  for  all  operator  tasks,  rather  than  a  small  subset  of  critical  tasks.  Perhaps  with 
additional  task  data,  it  would  be  possible  to  select  a  specific  occupation  or  set  of  occupations  as 
appropriate  for  performing  the  tasks. 
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7  Discussion 


This  section  discusses  the  models  created  in  IPME  and  ISMAT,  reviews  lessons  learned  from  the 
project,  and  provides  recommendations  for  future  work. 

The  same  UAV  scenario  was  modeled  in  IPME  and  ISMAT,  despite  the  differences  between  the 
two  tools  (see  Table  20).  It  was  challenging  to  export  the  model  from  IPME  and  import  it  into 
ISMAT  because  the  tools  use  different  file  formats.  Additional  export  functionality  was  added  to 
IPME  to  save  the  IPME  UAV  task  network  model  in  an  ISMAT-compatible  format.  Because  the 
ISMAT  crew  model  could  not  be  easily  configured  with  MO  SID  task  and  knowledge  statements, 
the  IPME  UAV  crew  model  was  not  imported  into  ISMAT.  Ideally,  if  ISMAT  supported  a  more 
configurable  crew  model,  functionality  could  be  added  to  IPME  to  export  the  crew  model  in  an 
ISMAT-compatible  format. 

Table  20:  IPME  and  ISMAT  Comparison 


IPME  and  ISMAT  IPME  and  ISMAT  Differences 

Similarities 

Task  network  structure 

User-defined  operator  characteristics  available  only  in  IPME; 
ISMAT  uses  Fleishman  taxonomy 

Simulation  engine 

Generate  different  outputs  (IPME  produced  JSI  values; 

ISMAT  produced  crew  skill  usage  information) 

Crew  work  representation 

ISMAT  uses  level  of  effort;  IPME  uses  workload 

The  method  of  incorporating  task,  skill,  and  knowledge  (TSK)  information  into  the  models 
differed  greatly.  Three  CSV  files  containing  task  ID,  goal  ID,  goal  descriptions,  and  duty  areas 
were  loaded  into  the  IPME  UAV  model  to  populate  simulation  variables  that  were  then  used  to 
calculate  the  JSI  value  for  operator-task  pairs.  For  the  ISMAT  model,  crew  members  with  titles 
related  to  UAV  missions  were  selected.  Fleishman  skills  were  selected  for  a  set  of  critical  tasks 
based  on  the  analyst’s  “best  guess.”  A  mapping  from  MOSID  TSK  statements  to  Fleishman  skills 
does  not  exist.  A  candidate  for  future  work  could  include  developing  this  mapping,  and  updating 
the  MOSID  TSK  statements  to  reflect  this  mapping. 

The  flexibility  of  IPME  is  useful,  but  the  effort  required  to  use  the  CSV  files  was  substantial  and 
required  the  help  of  a  software  developer.  To  simplify  this  step,  a  single,  consolidated,  fully 
populated  MOS  database  would  have  been  very  helpful.  With  this  database,  IPME  could  provide 
a  simple  interface  for  producing  a  list  of  TSK  statements. 

The  task  expression  logic  in  IPME  calculated  the  JSI  for  operator-task  pairs,  and  a  snapshot  was 
collected  to  record  those  JSI  values.  While  ISMAT  allows  user-configured  snapshots,  it  was  not 
possible  to  calculate  the  JSI  with  the  information  in  the  ISMAT  model.  This  forced  the  use  of  the 
built-in  ISMAT  reports  to  examine  skill  usage. 

Simplifying  the  IPME  model  would  have  made  importing  the  model  into  ISMAT  easier.  In  the 
future,  perhaps  it  would  be  better  to  use  only  one  part  of  the  scenario  (one  of  the  task  networks 
from  the  CMC  model),  and  avoid  using  continuous  tasks  as  they  have  different  representations 
and  possibly  different  underlying  execution  mechanisms  in  IPME  and  ISMAT. 
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In  conclusion,  the  TSK  statements  were  useful  in  calculating  the  JSI  because  the  JSI  does  provide 
some  insightful  qualitative  information.  The  addition  of  the  MOSID  information  did  make  a 
positive  contribution  to  the  UAV  model  in  IPME.  However,  an  argument  could  be  made  that  if 
the  Fleishman  skills  for  CF  AF  occupations  and  the  UAV  scenario  tasks  were  available,  the 
1SMAT  functionality  could  be  just  as  beneficial. 

Clearly,  the  incomplete  MOS  database  created  many  difficulties  in  this  project.  The  MOS  data 
and  the  database  structure  were  not  consistent.  Certain  occupational  traits  were  not  fully 
specified,  which  limited  the  validity  of  this  study.  However,  the  model  development  procedure, 
especially  the  use  of  occupational  knowledge  and  skills  attributes  in  a  human  performance  model, 
demonstrated  a  plausible  solution  for  addressing  personnel  and  manning  issues  through 
simulation.  Although  a  direct  contrast  of  simulation  predictions  between  IPME  and  ISMAT  was 
not  possible,  this  study  illustrated  two  different  modeling  approaches  for  addressing  manning 
requirements.  This  modeling  exercise  focused  on  these  differences  and  identified  a  number  of 
areas  in  the  current  IPME  implementation  that  demand  future  improvements.  The  list  below 
summarizes  changes  that  are  required  to  address  these  capability  gaps. 

•  It  is  important  to  provide  a  consistent  data  schema  for  storing  MOS  data.  This  is 
especially  true  for  critical  data  relationships. 

•  A  single  database  or  point  of  entry  (e.g.,  a  graphical  front-end)  will  reduce  the  effort  in 
MOS  data  searching  and  editing. 

•  It  is  computationally  easier  to  incorporate  quantitative  MOS  data,  rather  than  qualitative 
data,  into  a  human  performance  model.  Qualitative  data  were  not  directly  used  in  the 
models  developed  for  this  study. 

•  The  integrity  of  MOS  data  needs  to  be  further  validated.  For  example,  such  critical  data 
as  the  occupational  skill  statements  were  not  complete  in  the  existing  database.  Further, 
the  enumerated  data  types  were  not  consistent  within  a  data  set. 

•  It  requires  significant  effort  to  convert  a  model  between  applications.  As  we  observed  in 
this  study,  a  great  deal  of  time  was  spent  reformatting  the  IPME  task  data  for  the  ISMAT 
model.  The  adoption  of  a  common  schema  for  data  output  will  reduce,  but  not  eliminate, 
such  effort. 

•  There  exists  a  large  inventory  of  tasks,  skills  and  knowledge  (TSK)  statements  for  CF 
occupations.  To  choose  and  map  these  statements  to  a  specific  human  task  is  tedious  and 
requires  the  analyst  to  have  a  good  understanding  of  the  CF  occupational  specifications. 
Such  TSK  assignments  were  primarily  completed  on  a  manual  level  in  this  study.  It  is 
useful  to  automatize  the  process  and  reduce  the  effort  required  in  the  modeling  process. 
One  possible  solution  is  to  map  the  TSK  statements  to  a  generic  human  skill  taxonomy 
for  translating  task  requirements  into  occupational  TSK  statements. 

This  project  represents  our  effort  to  incorporate  military  occupational  data  into  human 
performance  models.  Occupational  attibutes  can  be  meaningfully  linked  to  operator  performance 
predictors.  However,  this  project  has  not  addressed  many  of  the  usability  issues  in  the  current 
implementation.  The  following  list  indicates  future  IPME  development  opportunities.  It  is 
weighted  according  to  priority  (highest  priority  first): 
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(1)  Incorporate  a  generic  human  skill  taxonomy  for  translating  human  task  requirements  into 
occupational  TSK  statements  (e.g.,  MOS  data); 

(2)  Create  a  TSK  data  collection  wizard  to  assist  in  the  specification  of  TSK  statements  in 
both  task  network  models  and  crew  models.  The  analyst  will  be  able  to  choose  which  TSK 
statements  apply  to  tasks,  and  allow  IPME  to  automatically  determine  which  operators  meet 
those  TSK  requirements; 

(3)  Generate  a  set  of  master  operators  with  generic  levels  of  TSK  statements  that  can  be  used 
in  models  as  benchmark  configurations; 

(4)  Since  the  manning  requirement  is  often  linked  to  specific  equipment  operation,  it  is  useful 
to  create  an  equipment  model  in  IPME  to  address  such  design  questions  as  maintenance 
requirements.  Conventional  performance  measures  (e.g.,  cognitive  workload)  can  also  be 
introduced  when  representing  operators’  interaction  with  the  equipment; 

(5)  A  Gantt  chart  view  will  enhance  the  visualisation  of  both  task  execution  and  personnel 
assignment. 

Traditionally,  modeling  and  simulation  (M&S)  have  been  applied  to  system  interface  design  and 
evaluation.  Although  this  is  important  for  determining  the  effectiveness  of  a  system,  a  major 
component  of  system  expense  is  attributed  to  the  manning  cost.  The  addition  of  occupational 
traits  in  human  performance  models  is  a  first  step  to  addressing  the  manning  requirement  in  a 
simulation  environment.  It  creates  a  bridge  to  link  M&S  to  the  human  resources  (HR)  cycle  of 
activities.  It  is  now  possible  to  create  an  IPME  model  that  describes  both  the  task  requirements 
(e.g.,  flying  a  UAV)  and  the  knowledge  and  skills  attributes  that  should  be  possessed  by  the 
operators.  Such  a  model  is  able  to  address  a  range  of  Human  Resource  (HR)  requirements,  such 
as  the  identification  of  training  requirements  and  their  prioritisation.  These  outputs  can  be  fed  into 
existing  HR  activities  for  identifying  a  potential  pool  of  candidate  operators,  formulating 
recruiting  plans,  and  developing  training  programs.  The  capability  developed  in  this  project 
represents  an  expansion  of  modeling  support  for  wider  HS1  domains.  This  project  has  further 
advanced  our  capability  of  using  M&S  for  covering  a  larger  portion  of  the  life  cycle  of  system 
development. 
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Annex  A  The  UAV  Mission 


The  mission  scenario  chosen  for  this  project  is  the  scenario  described  in  Kobierski’s  report 
“Hierarchical  Goal  Analysis  and  Performance  Modelling  for  the  Control  of  Multiple 
UAVs/UCAVs  from  an  Airborne  Platform,”  (DRDC  Toronto  CR  2004-063)  and  is  included  here 
for  reference.  The  timeline  was  consolidated  from  three  separate  tables  into  one  table  here. 

The  mission  scenario  described  in  this  annex  forms  the  basis  for  an  analysis  of  CP- 140 
Unmanned  Air  Vehicle  (UAV)  crewmember  activities  (goals  and  task  interaction).  This 
mission  is  set  seven  years  in  the  future. 

The  scenario  begins  with  a  fictitious  advanced  briefing  to  Commonwealth  Heads  of 
Government  Meeting  (CHOGM)  security  staff,  conducted  in  February  2011.  Following 
this,  a  description  of  the  situation  on  the  day  of  the  opening  of  the  CHOGM  is  provided, 
as  well  as  a  brief  overview  of  the  one-hour  scenario. . . . 

COMMAND  AND  CONTROL  OF  CP-140  UAV  TEAM 

The  CP- 140  is  manned  by  a  crew,  which  is  augmented  by  two  additional  crewmembers 
who  have  responsibility  for  controlling  UAV  assets.  These  are  the  UAV  Operator  (UAV 
Op)  and  the  UAV  Pilot  (UAV  Pit).  It  is  possible  that  these  individuals  may  be  CP- 140 
crewmembers  who  have  been  cross-trained  to  operate  the  Canadian  Forces  (CF)  UAV 
fleet.  The  CP- 140  in  under  the  control  of  the  Military  Operations  Centre  in  Halifax  and 
the  Medium  Altitude  Long  Endurance  (MALE)  UAV  is  under  the  control  the  Regional 
Operations  Centre  (ROC),  which  is,  for  the  purposes  of  this  scenario,  also  located  in 
Halifax.  The  CP- 140  tactical  crew  consists  of  the  Tactical  Navigator  (TACNAV),  or 
mission  commander,  the  UAV  Operator  and  the  UAV  Pilot.  The  TACNAV  occupies  his 
or  her  nonnal  position,  and  to  the  left  of  the  TACNAV  are  the  UAV  Operator  and  Pilot. 
These  two  individuals  occupy  the  Acoustic  Sensor  Operator  (ASO)  reconfigurable 
workstations  of  the  modernized  CP- 140.  The  TACNAV  has  overall  responsibility  for  the 
UAV  mission,  the  UAV  Operator  is  responsible  for  ensuring  that  the  data  and  images  are 
collected  effectively  and  controlled  for  use  in  civilian  court,  and  the  UAV  Pilot  (who  has 
the  least  complex  role)  is  responsible  for  UAV  flight  path  and  air  vehicle  systems. . . . 

A  BRIEF  OVERVIEW  OF  THE  SCENARIO 

For  the  purpose  of  this  scenario,  the  UAVs  are  controlled  from  the  CP- 140  Acoustic 
Sensor  Operator  (ASO)  positions,  and  the  UAV  crew  consists  of  the  TACNAV,  the  UAV 
Operator  occupying  the  ASO  1  position  and  a  UAV  Pilot  occupying  the  ASO  2  position 
(the  aisle  seat).  The  rest  of  the  CP- 140  crew  supports  the  mission.  The  scenario  begins  at 
18:00  hrs,  after  the  Aurora  crew  has  received  information  that  a  terrorist  threat  to  the 
CHOGM  is  possible  and  they  are  re-tasked  to  search  for  a  vessel  that  is  carrying  a 
container  the  size  of  the  KZO  launch  container  (approximately  10  ft  x  8  ft  x  20  ft). 
Reports  have  suggested  that  the  threat  may  come  from  a  trawler-sized  vessel.  The  vast 
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majority  of  the  fishing  fleet  is  operating  under  overcast  conditions,  approximately  100 
mn  offshore.  Visual  identification  will  require  flight  below  the  cloud  ceiling.  Two  of  the 
Canadian  Patrol  Frigate  (CPF)  VTUAVs  are  airborne  and  HMCS  Halifax  has  just  handed 
off  control  of  one  of  these  assets  (VTUAV  1)  to  the  CP- 140  flight  crew  to  assist  in 
contact  identification. 

Two  CF18s  are  scrambled  and  tasked  to  proceed  to  an  area  20  nm  south  of  St.  John’s  and 
mount  a  combat  air  patrol. 

Once  the  VTUAVs  clear  their  mother  ship,  HMCS  Halifax  makes  ready  and  launches  the 
new  MH.  The  mission  is  to  investigate  another  concentration  of  vessels  to  the  south.  The 
ship’s  crew  knows  that  recovery  of  VTUAV  1  will  be  necessary  at  approximately  the 
same  time  that  the  MH  returns,  but  the  situation  dictates  that  all  three  vehicles  be 
airborne.  Modification  of  the  flight  deck  to  accommodate  the  recovery  of  the  MH  and 
VTUAVs  is  the  latest  alteration  to  the  vessel. 

At  approximately  18:40  hrs,  contact  is  lost  with  VTUAV  1  as  it  approaches  a  vessel 
under  investigation.  The  Aurora  immediately  launches  three  Mini  UAVs  and  warns  other 
airborne  units  to  avoid  the  possible  threat.  The  MALE  UAV  (using  Inverse  Synthetic 
Aperture  Radar  [ISAR]  from  medium  altitude)  and  the  Mini  UAVs  investigate  the  vessel, 
and  at  the  same  time  HMCS  Halifax  makes  best  possible  speed  to  the  same  location. 
VTUAV  2  is  also  directed  towards  the  suspicious  boat  and  control  is  passed  to  the  CP- 
140  crew.  At  approximately  18:50  hrs,  a  Mini  UAV  transmits  an  image  of  men  working 
on  the  foc’sle  of  the  trawler.  These  individuals  have  exposed  a  large  storage  container. 
The  CP- 140  continues  to  covertly  observe  through  the  EO  sensor  of  VTUAV  2  and  Mini 
UAVs  as  the  container  is  opened  to  expose  a  Jet  Assist  Take-Off  (JATO)  UAV.  The 
CF18s  are  ordered  to  attack  the  now  identified  terrorist  boat.  Minutes  later,  the  Lethal 
UAV  is  launched. 

Communicating  with  the  MALE  UAV  operator  via  a  chat  room,  the  crew  provides  the 
departure  heading  of  the  Lethal  UAV  and  the  MALE  UAV  operator  initiates  tracking. 
Observation  of  a  second  Lethal  UAV  on  the  terrorist  boat  heightens  concern  until  a 
successful  fighter  attack  is  carried  out.  At  19:00  hrs,  the  scenario  ends,  although  the 
mission  is  still  ongoing.  The  Aurora  crew  initiate  a  search  for  the  Lethal  UAV,  which  is 
tracking  randomly  towards  St  John’s. 


Timeline 

Event 

18:00  hrs 

The  Aurora  UAV  crew  is  occupied  with  controlling  VTUAV  1,  which  is 
investigating  two  contacts  (Contacts  2  and  3);  at  the  same  time,  the  aircraft 
is 

transiting  to  observe  two  different  fishing  vessels  (Contacts  1  and  4). 

Initial  one- 

to-one  data  link  control  with  VTUAV  1  was  established  prior  to  18:00  hrs. 
The 
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CP- 140  is  level  at  4000  ft. 

18:01  hrs 

The  UAV  operator  re-familiarises  himself  with  the  systems  onboard 
VTUAV  1 ,  including  settings  established  by  the  CPF  UAV  crew.  As  a  test 
of  the  system,  he  initiates  an  observation  of  a  known  contact  in  the 
direction  of  flight.  The  crew’s  third  pilot  occupies  the  vacated  ASO  2 
position  and  reconfigures  this  workstation  to  a  STANAG  4586 
configuration  for  flight  control  of  the  CPF’s  VTUAV. 

18:02  hrs 

The  TACNAV  plans  the  route  for  the  Aurora,  and  requests  that  the  pilots 
descend  below  the  cloud,  and  turn  and  head  towards  the  first  contact  at 
high  speed.  Non-Acoustic  Sensor  operator  (NASO)  1  generates  a 
GENTRACK  on  the  target  and  directs  the  pilots  for  a  homing  from  the 
stern  until  the  EO  operator  gains  contact.  The  flight  crew  works  with  the 
NASO  1  (AESO  1)  who  is  using  the  onboard  radar  to  clear  the  way  ahead 
for  the  descent  and  at  the  same  time  search  for  the  contact  of  interest.  The 
NASO  2  configures  the  EO  to  begin  a  Charge  Coupled  Device  (CCD) 
colour  camera  and  an  Infrared  (IR)  sensor  search  to  classify  and 
potentially  identify  the  unknown  contact  once  the  aircraft  descends  below 
the  cloud. 

18:03  hrs 

The  NAVCOM  completes  his  radio  transmission,  acknowledging  the 
current  tasking  with  MOC  in  Halifax. 

18:04  hrs 

The  TACNAV  and  UAV  operators  work  as  a  team  to  plan  the  route  and 
search  activities  of  VTUAV  1 .  The  UAV  operator  is  occupying  the  ASO  1 
position  and  has  configured  the  workstation  to  a  STANAG  4586  standard 
for  control  of  imagery.  The  TACNAV  requests  that  the  UAV  Pilot  direct 
VTUAV  1  to  an  initial  heading  of  300°  true,  maintain  800  ft  Above  Sea 
Level  (ASL),  and  increase  speed  to  the  dash  speed  of  200  kts.  The  UAV 
Operator  uses  the  Telephonies  1700B  (Mark  II)  Radar  to  search  for 

Contact  2.  The  TACNAV  extrapolates  the  last  known  position,  course 
(315°  true)  and  speed  to  establish  a  datum.  Most  of  the  fishing  vessels  in 
the  area  are  headed  northwest  into  the  wind  with  their  nets  deployed. 

18:05  hrs 

NASO  1  is  fully  occupied  with  updating  the  radar  plot.  At  15  nm  from 
Contact  1,  the  Aircraft  Captain  (AC)  reports  that  the  aircraft  is  level  at  800 
ft. 

18:06  hrs 

The  NAVCOM  obtains  additional  information  regarding  the  type  of 
contact  and  potential  threat  the  crew  may  encounter,  updates  the 

Electronic  Support  Measures  (ESM)  library  and  presets  for  chaff  and 
flares. 
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18:07  hrs 

The  UAV  Pilot  begins  to  take  stock  of  the  fuel  onboard  VTUAV  1 , 
potential  landing  pads  (CPF,  Hibernia  oil  drilling  platforms,  etc.),  and 
refuelling  locations.  The  pilot  advises  that  the  UAV  will  have  at  least  two 
hours  on-task  and  that  a  more  refined  time  estimate  will  be  available  soon. 

18:08  hrs 

The  EO  Operator  establishes  contact  with  the  first  vessel  to  be 
investigated  (Contact  1 )  and  the  flight  crew  plans  an  EO  rig,  which  will 
allow  effective  observation  and  at  the  same  time  minimize  the  threat  from 
surface  to  air  missiles.  The  UAV  Pilot  and  the  TACNAV  conclude  that 
VTUAV  1  must  eventually  return  to  the  CPF  because  there  is  no  other 
suitable  landing  area  in  the  vicinity. 

18:09  hrs 

The  name  and  licence  of  Contact  1  are  determined. 

18:10  hrs 

The  flight  crew  reach  the  closest  point  of  prudent  approach  and  commence 
an  arc  around  Contact  1 .  Using  the  name  and  licence  of  Contact  1 ,  the 
NAVCOM  accesses  the  Department  of  Fisheries  and  Oceans  (DFO) 
online  database  for  the  East  Coast  to  get  a  confirmation  photo  and  history 
of  this  boat. 

18:11  hrs 

As  the  CP- 140  circles  Contact  1,  NASO  2  continues  to  observe  the  boat 
using  the  EO  suite. 

18:12  hrs 

The  UAV  Operator  and  TACNAV  continue  tracking  VTUAV  1  towards 
Contact  2.  While  generally  following  the  route  to  the  datum,  the  course  of 
VTUAV  1  is  varied  twice  to  allow  for  identification  of  two  contacts  near 
the  route.  NASO  2  reports  that  the  nets  on  Contact  1  are  just  about  to  be 
streamed,  and  a  large  box-shaped  object  on  the  foc’sle  is  a  large  pile  of 
crab  traps.  The  TACNAV  directs  the  crew  to  proceed  to  Contact  4.  With  a 
transit  speed  of  240  kts,  the  62  nm  distance  is  covered  in  approximately  16 
minutes. 

18:13  hrs 

MOC  advises  the  Aurora  crew  that  the  MALE  UAV  is  able  to  assist  in 
approximately  8  minutes.  The  crew  is  directed  to  contact  the  ROC.  The 
Raytheon  Sea  Vue  (Mark  II)  Radar  of  the  MALE  UAV  is  out  of  range. 

The  UAV  Operator  asks  Ordinance  to  prepare  one  Mini  UAV  for 
deployment.  The  TACNAV  sets  the  operating  height  electronically  for 

1 000  ft  so  that  the  tactical  crew  could  deploy  the  Mini  UAV  from  altitude 
and  with  minimum  time  delay  to  obtain  video  imagery  from  below  the 
clouds.  The  Mini  UAV  is  prepared  in  anticipation  of  deployment  over 
Contact  4.  The  AC  advises  that  Contact  1  has  been  identified  as  the  Guppy 
from  Portland,  Maine,  and  he  is  turning  the  aircraft  toward  Contact  4.  The 
TACNAV  requests  that  the  Aurora  Pilot  climb  to  4000  ft  in  order  to 
maintain  the  data  link  with  VTUAV  1 . 
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18:14  hrs 

The  NAVCOM  confirms  that  the  Guppy  is  not  on  any  of  the  Vessel  of 
Interest  lists. 

18:15  hrs 

The  CP- 140  continues  towards  Contact  4.  The  pilot  reports  that  the 
aircraft  is  level  at  4000  ft. 

18:16  hrs 

NASO  1  uses  the  ISAR  to  scrutinise  Contact  2.  It  is  determined  that 
although  the  length  and  breadth  of  Contact  2  match  the  search  criteria, 
there  is  nothing  unusual  about  the  structure  and  this  Contact  is  unlikely  to 
be  the  terrorist  vessel.  Regardless,  the  decision  is  made  to  identify  Contact 

2  using  VTUAV  1 .  VTUAV  1  changes  course  slightly  to  investigate 

Contact  2. 

18:17  hrs 

The  NAVCOM  requests  an  update  on  the  air  picture  from  the  Halifax. 
Contact  2  is  identified  as  the  Dolphin  from  Miami,  Florida.  It  is  fishing 
legally.  VTUAV  1  turns  towards  Contact  3  and  level  flight  at  800  ft  is 
maintained.  With  a  momentary  lull  in  the  action  as  both  the  CP- 140  and 
VTUAV  are  in  transit,  the  TACNAV  initiates  a  discussion  with  the  AC  to 
try  and  determine  the  location  of  the  terrorist  vessel.  It  is  deemed  that  a 
terrorist  vessel  would  have  turned  towards  St.  John’s,  the  site  of  the 
CHOGM.  At  the  same  time,  MOC  asks  the  CP- 140  to  also  investigate  a 
suspicious  boat  20  nm  north  of  VTUAV  1.  MOC  recommends  caution 
when  approaching  this  boat.  A  datum  for  Contact  5  is  plotted  on  the 
computer. 

18:18  hrs 

As  VTUAV  1  is  already  closer  to  the  new  target  vessel,  the  TACNAV 
directs  the  UAV  Pilot  to  investigate  Contact  5  before  Contact  3,  once 
investigation  of  Contact  2  is  complete. 

18:19  hrs 

VTUAV  1  has  completed  investigation  of  Contact  2  and  turns  towards 
Contact  5 .  The  VTUAV  1  time  on-task  is  approximately  2  hours  and  1 0 
minutes;  it  must  return  to  the  CPF.  Coordination  will  be  required  because 
that  will  be  the  approximate  time  that  the  MH  will  require  refuelling.  The 
deck  cycle  has  been  interrupted  due  to  the  urgent  nature  of  the  mission. 

The  remaining  VTUAV  2  time  on-task  is  lengthy  considering  the  recent 
launch,  and  will  not  require  refuelling  for  many  hours. 

18:20  hrs 

The  UAV  Operator  begins  to  create  a  surface  plot  of  the  area  around  the 
datum  as  VTUAV  1  transits  towards  the  new  search  area.  The  Aurora 
continues  towards  Contact  4  at  4000  ft.  ROC  advises  the  NAVCOM  that 
the  MALE  UAV  has  completed  the  search  in  his  area  and  is  standing  by  to 
assist  the  CP-140  on  the  eastern  boundary  of  the  Aurora’s  area. 

18:21  hrs 

The  TACNAV  requests  that  the  MALE  UAV  enter  the  CP- 140  area,  but 
remain  at  7000  ft  or  above,  whereas  the  CP- 140  will  remain  at  6000  ft  or 
below.  The  MALE  UAV  is  to  investigate  Contact  3. 
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18:22  hrs 

With  the  combined  Radar  data  from  VTUAV  1  and  the  Aurora,  the 
TACNAV  quickly  builds  a  plot  of  contacts  in  the  area  of  Contact  5  and 
selects  three  vessels  of  which  one  could  be  the  terrorist  boat.  The 

TACNAV  directs  VTUAV  1  to  investigate  the  first  of  the  three  boats  and 
requests  that  the  MALE  UAV  use  its  ISAR  radar  to  image  the  second  of 
the  three  boats  while  heading  to  Contact  3. 

18:23  hrs 

The  TACNAV  then  directs  his  attention  towards  Contact  4. 

18:24  hrs 

NASO  1  has  a  good  GENTRACK  on  Contact  4  and  directs  the  pilot  for 
the  homing  down  the  Mean  Line  of  Advance  (MLA). 

18:25  hrs 

The  UAV  Pilot  turns  VTUAV  1  towards  the  first  boat  (designated  Contact 
5)  and  then  concentrates  on  establishing  a  waypoint  near  this  boat,  which 
will  allow  a  discreet  approach.  The  pilot  is  aware  that  the  VTUAV  is 
relatively  noisy  and  an  approach  from  east-southeast  will  keep  the  vehicle 
downwind.  He  also  considers  that  the  overcast  conditions  will  give  no 
advantage  to  approaching  with  the  sun  behind  the  vehicle.  The  UAV  Pilot 
decides  that  a  waypoint  on  a  relative  bearing  of  120°  true  at  5  nm  from  the 
contact  is  best,  although  this  requires  a  diversion  from  the  direct  route  and 
a  90  left  turn  to  approach  the  boat.  The  UAV  Pilot  notes  that  the  boat  is 
heading  west-northwest  like  the  other  fishing  vessels.  The  UAV  Pilot 
established  a  transit  speed  of  120  kts. 

18:26  hrs 

The  TACNAV  presets  the  waypoints  and  search  pattern  for  the  Mini-UAV 
(automatic  actions  to  be  taken  after  launch  if  the  UAV  Pilot  cannot  gain 
control  of  the  vehicle). 

18:27  hrs 

The  UAV  Pilot  brings  VTUAV  1  as  high  as  possible  without  entering  the 
clouds. 

18:28  hrs 

After  reviewing  the  Contact  history  (from  the  original  portion  of  the 
fisheries  patrol),  the  TACNAV  advises  the  UAV  Pilot  and  UAV  Operator 
that  one  of  the  three  boats  in  that  group  should  be  the  fishing  boat  “Trust 
Me”. 

18:29  hrs 

The  AC  tells  the  TACNAV  that  the  aircraft  is  two  minutes  back  from 
Contact  4  and  he  is  slowing  the  aircraft  down  to  a  speed  that  will  allow 
deployment  of  a  Mini  UAV.  Using  the  estimates  of  the  wind  generated  by 
the  Embedded  Global  Positioning  Systems/Inertial  Navigation  Systems 
(EGIs),  the  TACNAV  makes  a  final  determination  of  the  Pickle  Point  and 
passes  this  to  the  pilot. 
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18:30  hrs 

The  ROC  advises  that  the  ISAR  imagery  of  the  second  boat  should  be 
available  in  two  minutes. 

18:31  hrs 

The  flight  deck  crewmembers  review  the  online  checklist  for  Airborne 

Mini  UAV  Deployment. 

18:32  hrs 

The  UAV  Operator  reports  to  the  TACNAV  and  UAV  Pilot  that  he  has 
verified  the  identity  of  the  “Trust  Me”  from  Come-By-Chance, 
Newfoundland,  and  recommends  that  the  pilot  resume  course  to  the  third 
boat  suspected  to  be  Contact  5 .  The  UAV  Pilot  provides  VTUAV  1  with 
commands  to  head  towards  a  waypoint  established  at  5  nm  from  the 
unknown  boat.  The  UAV  Pilot  verifies  that  VTUAV  1  is  operating  within 
normal  parameters  and  prepares  to  take  control  of  the  Mini  UAV  once  it 
has  dropped  to  1000  ft,  is  deployed  from  the  A-size  parachute  stabilised 
container,  and  is  established  in  straight  and  level  flight.  The  initial  heading 
will  be  into  the  wind,  as  pre-programmed  prior  to  deployment.  NASO  2 
accesses  the  ROC  image  server,  extracts  the  MALE  UAV  ISAR  image 
and  compares  it  to  the  last  VTUAV  contact.  He  advises  the  TACNAV  that 
both  the  MALE  UAV  and  VTUAV  1  have  investigated  the  same  boat.  The 
NAVCOM  advises  the  ROC  of  the  mix-up  and  requests  imaging  of  the 
second  boat. 

18:33  hrs 

Upon  reaching  the  UAV  Fly-to-Point,  the  CP- 140  automatically  releases 
the  Mini  UAV.  The  AC  verbally  advises  that  the  Mini  UAV  has  been 
deployed  over  Contact  4,  and  both  the  TACNAV  and  UAV  Pilot  verily 
this  with  their  display  symbology.  The  Aurora  is  turned  towards  the  area 
of  Contact  5  to  assist  with  finding  this  boat.  The  crew  track  the  descent  of 
the  Mini  UAV  over  Contact  4  and  wait  for  an  uplink  of  a  video  image.  On 
queue,  at  1000  ft,  the  Mini  UAV  transmits  that  it  has  established  level 
flight,  into  wind,  and  is  standing  by  for  control  signals.  The  UAV  Pilot 
takes  control  of  the  Mini  UAV  and  verifies  its  serviceability. 

18:34  hrs 

The  TACNAV  directs  the  AC  to  maintain  the  current  altitude  (4000  ft). 

The  UAV  Pilot,  with  assistance  from  NASO  1 ,  turns  the  Mini  UAV  in  the 
direction  of  Contact  4  and,  while  accommodating  for  the  wind,  establishes 
a  track  directly  towards  the  boat.  He  also  initiates  an  enroute  descent  to 

300  ft.  At  this  time,  he  checks  on  the  progress  of  VTUAV  1  and  finds  that 
the  vehicle  is  proceeding  as  directed  towards  the  waypoint.  NASO  1 
advises  that  the  Mini  UAV  is  within  2  nm  of  the  contact,  and  at  the  same 
time,  the  UAV  Operator  advises  that  EO  is  active,  although  the  boat  is  not 
in  the  Field-Of-View  (FOV). 
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18:35  hrs 

NASO  2  reports  to  the  TACNAV  that  the  MALE  UAV  imaging  of  the 
second  boat  in  the  vicinity  of  the  new  datum  has  been  completed. 

Although  the  boat  could  not  be  identified,  it  was  determined  to  be  a  small 
trawler  and  there  did  not  appear  to  be  any  container  shaped  structure  on 
the  decks.  The  TACNAV  rejects  this  boat  as  a  possible  terrorist  threat  and 
directs  the  MALE  UAV  to  attempt  imaging  of  the  third  boat.  Having 
established  that  the  Mini  UAV  deployed  over  Contact  4  is  functioning 
nominally,  the  UAV  Pilot  passes  control  of  the  flight  path  to  the  UAV 
operator,  but  still  maintains  an  overview  of  the  UAV  progress  to  assist  if 
required.  Manual  flight  control  is  a  reversionary  mode,  however,  the  pilot 
would  have  to  have  been  following  the  UAV  activities  in  order  to  use  this 
reversionary  mode  effectively,  if  it  becomes  necessary.  The  UAV  Pilot 
also  monitors  the  progress  of  VTUAV  1 .  The  UAV  Operator  uses  the 
point  and  click  control  features  of  the  interface  to  establish  a  sea  level 
waypoint  at  the  location  of  Contact  4.  The  crew  does  not  have  to  concern 
themselves  with  the  data  link  because  the  Mini  UAV  transmits  omni¬ 
directionally.  With  the  UAV  FOV  presented  on  the  UAV  Operator’s 
tactical  plot  (tacplot),  the  operator  drags  the  FOV  trapezoid  over  the  radar 
contact  and  zooms  in  using  the  UAV’s  EO  suite.  The  Mini  UAV  flies 
directly  towards  the  unsuspecting  fishers. 

18:36  hrs 

The  UAV  Operator  records  a  high-quality  video  image  of  Contact  4.  He 
uses  the  onboard  database  and  support  from  the  ROC  database  to 
determine  that  the  boat  is  a  known  vessel,  "Master",  but  is  fishing  illegally 
in  a  zone  that  does  not  open  for  another  5  hours  and  27  minutes.  The  crew 
has  just  hauled  in  the  nets  and  the  rear  deck  is  awash  in  fish.  The 

TACNAV  requests  that  NASO  1  update  the  position  of  Contact  3  and  all 
three  boats  in  the  vicinity  of  Contact  5.  The  TACNAV  then  directs  the 
NAVCOM  to  access  the  online  “Fishing  Violation”  report  and  action  it. 

18:37  hrs 

The  Mini  UAV  over  Contact  4  is  set  on  autonomous  Operations  to  report 
automatically  all  information  required  to  prosecute  the  vessel  "Master"  in 
court.  Video  and  location  information  is  recorded  onboard  the  Aurora.  The 
Aurora  crew  direct  their  attention  to  the  remaining  unknown  boat  in  the 
vicinity  of  the  new  datum,  which  is  determined  (by  the  process  of 
elimination)  to  be  Contact  5  and,  thus,  has  a  higher  priority  than  the 
fishing  violator. 

18:38  hrs 

The  crew  determines  that  with  the  approach  path  planned,  VTUAV  1  will 
pass  near  a  small  fishing  boat  at  approximately  2  miles  from  Contact  5 . 

The  crew  decides  that  they  will  observe  this  boat  enroute  to  their  primary 
objective.  The  CPF  advises  that  VTUAV  2  is  investigating  a  small  group 
of  fishing  vessels  to  the  south  of  the  VTUAV  1  position  (which  they  know 
from  the  LINK- 1 1  picture  transmitted  by  the  CP- 140).  The  MALE  UAV 
completes  imaging  of  Contact  3  and  attempts  to  image  Contact  5 . 
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18:39  hrs 

NASO  2  reports  to  the  TACNAV  that  the  ISAR  imagery  of  Contact  3 
from  the  MALE  UAV  shows  that  it  is  very  similar  to  the  previous  boat 
studied  and  that  there  were  no  suspicious  container  shapes  on  the  decks. 

The  TACNAV  decides  that  further  investigation  of  Contact  3  is  not 
necessary  and  turns  to  the  surface  plot  to  determine  how  to  employ  the 
MALE  UAV.  He  begins  to  resize  the  area  displayed  on  his  workstation. 

The  UAV  Operator  reviews  the  data  being  collected  by  the  Mini  UAV 
deployed  earlier  and  finds  that  good  video  was  collected  up  until  the 
fishers  spotted  the  Mini  UAV  and  then  covered  the  deck  with  a  tarpaulin. 
The  boat  had  turned  south  with  the  Mini  UAV  in  a  holding  pattern 
approximately  Vi  nm  aft.  The  NAVCOM  reports  the  fishing  violator  to  the 
DFO,  and  advises  where  video  and  location  information  can  be  found  via 
a  secure  chat  room.  The  UAV  Pilot  monitors  the  progress  of  VTUAV  1  as 
it  flies  towards  the  waypoint. 

18:40  hrs 

VTUAV  1  reaches  the  waypoint  and  turns  towards  Contact  5 .  The  EO 
sensor  shows  only  the  small  fishing  boat  3  miles  from  Contact  5 . 

18:41  hrs 

At  three  miles  from  Contact  5,  the  VTUAV  maintains  800  ft  to  observe 
the  object  of  their  search.  The  UAV  Operator  then  swings  the  EO  over  to 
photograph  the  fishing  boat  they  are  passing.  The  fishers  onboard  are 
looking  and  pointing  at  the  UAV.  The  ROC  reports  to  the  NAVCOM  that 
the  ISAR  imagery  from  Contact  5  shows  a  very  unusual  structure  on  the 
bow  of  the  vessel.  Strong  returns  are  obtained  from  large  metal  comers 
that  are  not  common  on  the  other  fishing  boats.  The  image  of  the  small 
fishing  boat  is  the  last  image  recorded  when  the  UAV  Pilot  reports  that 
VTUAV  1  has  stopped  transmitting  data.  Moments  later  NASO  1  switches 
the  radar  to  Air-to-Air  and  reports  that  radar  contact  with  VTUAV  1  has 
been  lost. 
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18:42  hrs 

The  TACNAV  contacts  the  ROC  and  requests  that  the  MALE  UAV  keep 
imaging  contact  5 .  The  TACNAV  quickly  realizes  that  someone  must  take 
control  of  the  situation,  the  Mini  UAVs  onboard  the  Aurora  and  VTUAV 

2  are  the  only  source  of  additional  information.  The  MH  has  been  vectored 
towards  the  scene,  but  will  not  arrive  for  at  least  25  minutes.  The  crew 
assesses  that  they  are  the  most  capable  platform  on  the  scene  at  the  current 
time.  The  CP- 140  assumes  Shipbome  Aircraft  Controller  (SAC)  duties. 

The  CPF  advises  that  it  can  vector  VTUAV  2  towards  Contact  5  and 
complete  a  handover  to  the  CP-140.  The  VTUAV  is  20  nm  from  Contact  5 
and  the  CPF  advises  that  the  CP- 140  crew  should  take  control  of  this  asset 
in  one  minute.  MOC  also  advises  that  the  CF-18s  have  been  tasked  to 
close  at  maximum  speed.  This  section  of  hornets  is  to  report  momentarily 
on  their  weapons  load  and  weapon  of  choice  to  render  Contact  5 
incapacitated  on  the  designated  frequency.  The  NAVCOM  sets  the 
appropriate  radio.  The  CP- 140  crew  is  unclear  as  to  why  the  VTUAV  was 
lost.  A  review  of  the  last  frames  of  video  does  not  help,  because  the  small 
fishing  boat  the  crew  was  observing  at  the  time  did  not  appear  to  take  any 
overt  action.  The  TACNAV  determines  that  identification  of  Contact  5 
must  be  performed  as  soon  as  possible  to  avoid  sinking  a  potentially 
neutral  boat.  The  TACNAV  advises  both  the  UAV  Operator  and  the  UAV 
Pilot  to  terminate  the  Mini  UAV  over  Contact  4. 

18:43  hrs 

The  TACNAV  requests  that  the  AC  increase  speed  to  Velocity  Never  To 
Exceed  (VNE)  or  'go  Buster'.  The  current  altitude  (4000  ft)  will  keep  the 
aircraft  above  cloud  and  hidden  from  EO/IR  missiles.  Transmissions  are  to 
be  secure  from  this  point  on.  Full  Electronic  Warfare  (EW)  policies  are  to 
be  adhered  to.  The  TACNAV  briefs  the  crew  that  he  intends  to  over  fly 
the  contact  and  deploy  three  Mini  UAVs  and  then  establish  an  orbit,  which 
would  maximize  reception  from  the  Mini  UAVs.  He  requests  that  the 
NAVCOM  transmit  these  intentions  to  the  CF-18s  and  the  MOC. 
Preliminary  imaging  data  are  received  from  the  MALE  UAV,  which  show 
a  medium  sized  fishing  vessel  with  an  unusual  structure  near  the  bow.  The 
boat  is  maintaining  a  steady  course  north-northeast. 

18:44  hrs 

The  pilot  advises  the  crew  that  the  aircraft  is  2  minutes  back.  The  crew 
takes  control  of  VTUAV  2  and  vectors  it  directly  towards  Contact  5,  at  an 
altitude  of  2500  ft  (above  the  clouds).  Estimated  Time  of  Arrival  (ETA)  at 
Contact  5  is  8  minutes.  The  TACNAV  asks  Ordnance  to  load  three  Mini 
UAVs,  and  set  them  to  establish  level  flight  at  1000  ft,  800  ft  and  600  ft. 

He  then  asks  the  UAV  Pilot  to  use  the  two  lowest  Mini  UAVs  to  initiate 
cloverleaf  patterns  about  the  contact  with  a  descent  to  200  ft  and  400  ft  on 
each  pass.  The  highest  Mini  UAV  is  to  be  used  for  a  circular  pattern  about 
the  contact  at  800  ft.  Wind  and  mutual  separation  must  be  accommodated, 
however,  maximizing  viewing  time  is  essential.  The  UAV  Operator 
advises  that  he  will  transmit  all  relevant  photos  and  video  images  via  the 
net  chat  room. 
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18:45  hrs 

The  UAV  Pilot  recommends  to  the  TACNAV  that  a  Mini  UAV  be 
deployed  to  investigate  the  site  where  contact  was  lost  with  VTUAV  1 . 

The  TACNAV  concurs,  puts  a  waypoint  on  the  Electronic  Horizontal 
Situation  Indicator  (EHSI)  and  requests  the  AC  to  over-fly  this  waypoint 
after  deploying  the  first  three  Mini  UAVs.  Ordnance  advises  that  another 
Mini  UAV  for  a  600  ft  search  altitude  is  being  prepared.  NASO  1 
conducts  a  radar  search  of  the  area,  finds  a  weak  contact  near  the  point 
that  VTUAV  1  was  lost,  and  provides  this  new  information  to  the 
TACNAV,  who  updates  the  waypoint  provided  to  the  pilot  moments 
before.  The  AC  slows  the  CP- 140  down  to  Mini  UAV  launch  speed. 

18:46  hrs 

The  TACNAV  deploys  three  Mini  UAVs  at  five-second  intervals.  The  AC 
then  turns  the  aircraft  towards  the  next  waypoint  and  advises  “one  minute 
back”. 

18:47  hrs 

A  fourth  Mini  UAV  is  deployed  as  the  Aurora  reaches  the  assigned 
waypoint  over  the  downed  VTUAV.  The  first  three  Mini  UAVs  appear  on 
the  tacplot  within  10  seconds  of  each  other  and  start  transmitting  video 
and  data.  The  Hornets  check  in  with  the  Aurora  and  advise  that  they  are 
prepared  to  attack  the  vessel  with  precision  guided  weapons.  The 

NAVCOM  advises  the  fighters  to  stand-by  while  visual  confirmation  of 
the  threat  is  acquired. 

18:49  hrs 

The  Mini  UAVs  show  images  of  a  fishing  boat  transiting  in  the  direction 
of  the  swell  and  across  the  wind  lines.  The  relative  wind  is  at  90°  off  the 
bow  to  port. 

As  the  Mini  UAVs  approach  the  boat,  men  can  be  seen  moving  on  the 
deck  and  a  container  is  shown  on  the  bow  of  the  boat.  The  Mini  UAV  that 
was  launched  over  the  downed  VTUAV  reports  that  it  has  established 
straight  and  level  flight  and  is  awaiting  instructions.  The  UAV  Pilot  tasks 
this  Mini  UAV  with  a  circular  search  pattern  about  the  intermittent  radar 
contact  at  the  same  location.  The  UAV  Operator  notes  the  instructions  and 
opens  a  fifth  viewing  window  to  monitor  imagery  of  the  downed  VTUAV. 

18:50  hrs 

The  UAV  Pilot  takes  control  of  the  first  Mini  UAV  at  the  scene  and  using 
the  controls  available  to  him,  breaks  off  the  cloverleaf  pattern  to  fully 
investigate  the  structure  on  the  boat.  The  Aurora  crew  recognizes  a 
medium  range  JATO  UAV  mounted  inside  a  transportation  container. 

Both  sides  of  the  container  are  open  and  men  are  removing  flags  and 
battens  from  the  UAV.  The  CP- 140  crew  also  observes  what  appears  to  be 
a  second  container  mounted  mid-ships.  This  structure  is  not  clearly 
visible. 
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18:51  hrs 

The  TACNAV  and  UAV  Operator  immediately  realize  that  this  is  the 

Lethal  UAV  they  are  searching  for  and  that  the  JATO  UAV  is  pointing 
into  the  wind  and  is  being  readied  for  launch.  It  appears  that  the  personnel 
on  the  vessel  are  aware  that  they  are  being  observed  and  a  sense  of 
urgency  demonstrated  by  their  actions  indicates  that  they  know  that  time  is 
of  the  essence,  considering  their  overt  actions  against  VTUAV  1 .  The 

UAV  Operator  scrutinizes  the  suspicious  structure  mid-ships  concludes 
that  it  is  probably  a  second  Lethal  UAV  container. 

18:52  hrs 

The  TACNAV  asks  the  UAV  Pilot  if  it  would  be  possible  to  crash  a  Mini 
UAV  into  the  container.  The  answer  is  that  the  control  interface  and 
associated  time  delays  would  make  it  impossible  to  hit  this  moving  target. 
However,  use  of  the  laser  designator  on  board  a  Mini  UAV  would  allow 
targeting  for  a  missile  from  a  CF-18.  The  TACNAV  responds  by  issuing  a 
Flash  message  requesting  an  immediate  attack  by  the  flight  of  Hornets. 

The  TACNAV  requests  that  the  CP- 140  over- fly  the  boat  once  more  at 

4000  ft  to  accommodate  the  launch  of  the  laser  designator  Mini  UAV. 
Although  these  Mini  UAVs  are  much  more  expensive  than  the  standard 

EO  Mini  UAVs,  it  is  clear  to  the  TACNAV  that  the  crew  must  have  one 
available  on  an  as  required  basis,  and  the  time  required  to  deploy  this 
device  can  be  saved  by  launching  it  now.  The  TACNAV  requests  that  the 
laser  designator  Mini  UAV  be  put  into  a  holding  pattern  about  a  waypoint 
he  inserts  on  the  tacplot  east  of  Contact  5 .  The  Hornet  Lead  confirms  the 
instructions  and  pre-positions  for  the  attack.  The  NAVCOM  instructs  the 
MALE  UAV  to  remain  in  the  southwest  sector  of  the  area  until  the 
fighters  have  returned  to  Combat  Air  Patrol  (CAP)  altitude. 

18:53  hrs 

The  TACNAV  advises  NASO  1  that  the  Lethal  UAV  may  be  launched  at 
any  time,  and  that  they  are  to  track  the  UAV  for  as  long  as  possible. 

NASO  1  switches  the  radar  to  Air-to-Air  mode.  The  AC  suggests  that 
once  the  Lethal  UAV  is  clear  of  the  terrorist  boat,  they  could  establish  a 
trail  formation  and  visually  track  the  UAV  inbound  to  St.  John’s.  At  the 
same  time,  he  advises  the  crew  that  the  aircraft  is  one  minute  back  from 
the  launch  point  for  the  laser  designator  Mini  UAV. 

18:54  hrs 

The  laser  designator  Mini  UAV  is  launched  and  the  CP- 140  retreats  to  a 
safe  distance  from  the  terrorist  vessel.  The  tacplot  indicates  the  position  of 
the  newest  Mini  UAV. 

18:55  hrs 

The  UAV  Pilot  establishes  control  of  the  laser  designator  Mini  UAV  and 
directs  it  to  a  holding  pattern  at  the  TACNAV  established  waypoint.  The 
laser  designator  is  slaved  to  Contact  5.  At  the  same  time,  the  TACNAV 
maintains  reconnaissance  of  the  terrorist  vessel  using  the  previously 
launched  Mini  UAVs.  VTUAV  2  is  directed  southeast  of  Contact  5,  to 
hold  away  from  the  Hornet’s  intended  track. 

18:56  hrs 

The  Lethal  UAV  is  launched  from  the  terrorist  boat.  This  is  observed  by 
the  CP- 140  UAV  crew.  NASO  1  commences  tracking  of  the  Lethal  UAV, 
but  loses  contact  shortly  after  it  turns  to  the  north.  A  Flash  Message  is  sent 
to  MOC  via  SATCOM. 
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18:57  hrs 

The  crew  realize  that  capture  of  the  terrorist  would  be  beneficial;  however, 
they  are  concerned  about  the  suspicious  container  remaining  unidentified 
on  the  boat.  At  the  same  time,  the  fighters  report  that  they  will  be  ready  to 
commence  an  attack  in  one  minute.  The  TACNAV  requests  that  the 
fighters  continue  but  not  arm  their  weapons.  At  the  same  time,  the 
TACNAV  requests  more  information  from  the  UAV  Operator.  The  UAV 
Operator  has  turned  to  Mini  UAVs  and  is  diving  them  towards  the  boat. 

The  UAV  Pilot  has  descended  VTUAV  2  below  cloud,  and  is  observing 
the  boat  from  a  range  of  3  miles  downwind.  The  video  clearly  shows  that 
the  terrorists  are  readying  a  second  Lethal  UAV  and  the  TACNAV  directs 
the  Hornets  to  “go  hot”.  Although  the  fighter  pilots  are  using  the  “big  sky” 
approach  to  collision  avoidance,  the  CP- 140  crew  dive  the  Mini  UAVs  to 
just  above  the  wave  tops  to  avoid  a  midair  collision.  The  VTUAV  anti¬ 
collision  strobe  light  is  set  to  ON  as  the  fighters  pass.  Although  the  crew 
havs  lost  radar  contact  with  Lethal  UAV,  they  believe  that  the  only  course 
of  action  is  to  begin  a  search  for  this  UAV. 

18:58  hrs 

The  terrorist  boat  is  destroyed  with  one  GBU-16,  a  1000  lb  Laser  Guided 
Bomb  (LGB).  The  TACNAV  begins  the  process  of  setting  up  a  search 
pattern  for  the  Lethal  UAV  while  the  UAV  Operator  sets  the  Mini  UAVs 
to  self-directed  mode. 

18:59  hrs 

The  CP- 140  UAV  crew  coordinates  with  the  CPF  UAV  Controller  for  the 
return  of  VTUAV  2.  Although  the  MALE  UAV  is  able  to  provide  an 
initial  track  for  the  Lethal  UAV,  it  also  loses  contact.  The  scenario  ends. 

EPILOGUE 

Based  on  direction  provided  by  the  CP- 140  crew,  the  Modernized  CF-18  aircraft  shot 
down  the  Lethal  UAV,  which  was  en  route  to  St.  John’s  harbour.  The  target  was 
determined  to  be  the  British  destroyer.  The  RCMP  apprehended  two  individuals  who 
were  erecting  a  laser  designator  on  the  slopes  of  Signal  Hill.  This  laser  designator  would 
have  provided  final  control  signals  to  the  Lethal  UAV  to  ensure  maximum  damage. 
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Annex  B  Goal  ID,  OSD  ID,  and  Task  ID  Mappings 


Goal  ID  and 

Completion 

OSD 

OSD 

CMC 

OSD 

CMC 

Notes 

Goal 

Time  (sec) 

ID 

Operator 

Scenario 

Page 

IPME 

Descriptor 

Segment 

Number 

Model 

Task  ID 

All  Operators 

2.6.3 

identification 
of  known 
terrorist  units 

7.2.7  MALE 
UAV  systems 
are  monitored 
and  managed 

MC 

2.1.1  radar  plot 

4-9 

164 

TN 

2 

2 

21.12.11 

of  an  area  of 

166 

TN 

2 

2 

21.12.12 

interest  is 

current 

167 

UP 

3 

7 

27.6.5 

3.5.1  command 
&  control  of 
tactical 
situation  is 
assessed 

12 

192 

TN 

3 

10 

27.12.1 

3.5.2  a 

determination 
of  the  SAC 
(OSC)  is 
completed 

12 

196 

TN 

3 

10 

27.12.3 

5.1.3  relevant 
rules  of 

8 

152 

TN 

1 

2 

4.1.13.5 

engagement 
are  reviewed 

5.2.1  overt 

3 

490 

UO 

3 

42 

27.29.1 

actions  of 

491 

TN 

3 

42 

27.29.12 

terrorist  unit 
personnel  are 
observed 

492 

UP 

3 

43 

27.29.13 

5.3.1  potential 
weapons 
onboard 
terrorist  unit 

6 

414 

TN 

3 

30 

27.20.11 

5.3.8  risk  to 
neutral  units 

6 

218 

TN 

3 

6 

27.6.17 
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Goal  ID  and 

Completion 

OSD 

OSD 

CMC 

OSD 

CMC 

Notes 

Goal 

Time  (sec) 

ID 

Operator 

Scenario 

Page 

IPME 

Descriptor 

Segment 

Number 

Model 

Task  ID 

5.5.2  selection 

5-18 

436 

TN 

3 

34 

27.23.14 

of  best  units  to 
counter  to 
terrorist  threat 

526 

TN 

3 

46 

27.32.1 

5.5.6  selection 

5-20 

420 

TN 

3 

30 

27.20.8 

of  best 

465 

UO 

3 

38 

27.27.7 

offensive 

466 

TN 

3 

38 

27.27.8 

system(s) 

467 

UP 

3 

39 

27.27.9 

500 

TN 

3 

42 

27.29.7 

541 

TN 

3 

46 

27.32.12 

5.6.3  tasking 

6-9 

538 

TN 

3 

50 

27.33.11 

message  has 

550 

TN 

3 

46 

(task  name 

been 

transmitted 

is  wrong) 
27.33.4 

5.7.8  damage 
report  provided 
to  friendly  unit 

8 

567 

TN 

3 

50 

27.34.10 

6.4  terrorist's 

potential 

location 

5-30 

6.8.1  the  need 
for 

contingency 
plans  is 
addressed 

30 

120 

TN 

3 

2 

27.2.1 

6.8.2 

contingency 
plans  are 
created 

10 

150 

TN 

1 

2 

4.1.13.2 

6.8.3 

contingency 
plans  are 
discussed 

10 

151 

TN 

1 

6 

4.1.13.9 

7.3. 1.2  Mini 

5-15 

340 

TN 

1 

22 

4.3.9.10 

UAV  search 

206 

TN 

2 

10 

21.18.1 

pattern  is 

260 

UP 

2 

11 

21.19.2 

planned 

262 

TN 

2 

10 

21.19.3 

354 

UP 

2 

27 

21.21.17 

235 

TN 

3 

10 

27.11.8 

72 


DRDC  CR  2009-059 


Goal  ID  and  Completion  OSD  OSD 

Goal  Time  (sec)  ID  Operator 

Descriptor 

CMC  OSD 

Scenario  Page 
Segment  Number 

CMC  Notes 

IPME 

Model 

Task  ID 

7.3.1.10  crew 
is  briefed  on 
use  of  Mini 
UAVs 

18-20 

236 

542 

561 

TN 

TN 

TN 

3 

3 

3 

10 

46 

46 

27.11.17 

27.32.6 

27.34.2 

7. 3. 1.1 1  crew 
have  been 
requested  to 
launch  Mini 
UAV 

8 

452 

TN 

3 

34 

27.25.10 

7.3.8. 1 
previous 
minutes  of 
video  are 
reviewed 

20 

588 

TN 

2 

46 

21.35.11 

9. 3. 2.2  request 
for  surface  plot 
is  passed  to 

ROC 

5 

304 

TN 

1 

14 

4.2.7.10 

9. 3. 2. 3  fighter 
aircraft  are 
directed  to 
attack  threat 

14 

438 

TN 

3 

34 

27.24.5 

9.4.1.5 

information  of 
a  general 
nature 

3-35 

124 

TN 

3 

2 

27.2.4 

PO 

2.1.4  tactical 
plot  icons  are 
current 

8 

360 

442 

UO 

uo 

1 

1 

18 

30 

4.2.16.10 

21.23.11 

5.6.4  tasking 
message  has 
been 

acknowledged 

9 

537 

539 

UO 

UP 

3 

3 

50 

51 

27.33.10 

27.33.12 

7. 1.4.2 

VTUAV  EO 
sensor  is  used 
for  a  test 
observation 

10 

168 

uo 

1 

10 

4.1.14.16 

7. 1.5.6 

VTUAV  EO 
images  are 
monitored 

25 

434 

UP 

2 

19 

21.23.17 
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Goal  ID  and 

Completion 

OSD 

OSD 

CMC 

OSD 

CMC 

Notes 

Goal 

Time  (sec) 

ID 

Operator 

Scenario 

Page 

IPME 

Descriptor 

Segment 

Number 

Model 

Task  ID 

7. 1.9. 3  initial 

10 

164 

UP 

1 

11 

4.1.14.17 

system  settings 
on  VTUAV 
are  reviewed 

166 

UO 

1 

6 

4.1.14.9 

7.3. 7.5  Mini 

14 

223 

UO 

3 

10 

27.8.7 

Not  in  trc 

UAV  image 
file  is  stowed 

9.4.1.3 

3-20 

362 

UO 

1 

18 

4.2.16.15 

identification 

392 

UO 

1 

30 

4.2.28.7 

and  activities 

444 

UO 

1 

30 

4.2.26.2 

of  contact 

424 

UO 

1 

26 

4.3. 8.1 

268 

TN 

2 

6 

21.17.21 

*these  OSD 

296 

UP 

2 

15 

21.20.4 

IDs  refer  to 

308 

UO 

2 

14 

21.20.11 

goal  9.4. 1.5  in 

366 

UO 

2 

14 

21.22.5 

the  OSD 

390 

UO 

2 

22 

21.24.11 

469 

TN 

2 

30 

21.24.23 

418 

UP 

2 

31 

21.28.4 

436 

UO 

2 

18 

21.23.6 

438 

UP 

2 

19 

21.23.8 

442 

UO 

2 

18 

21.23.11 

454 

UO 

2 

30 

21.29.8 

502 

UO 

2 

34 

21.31.4 

514 

UO 

2 

38 

21.31.12 

524 

UO 

2 

42 

21.31.23 

527 

UP 

2 

43 

21.31.25 

530 

UO 

2 

42 

21.31.27 

536 

UO 

2 

42 

21.34.2 

555 

UO 

2 

38 

21.32.13 

560 

TN 

2 

38 

21.32.17 

561 

UO 

2 

38 

21.32.18 

617 

UO 

2 

50 

21.38.11 

623 

TN 

2 

50 

21.38.15 

624 

UO 

2 

50 

21.38.16 

177 

TN 

3 

6 

27.5.19 

410 

UO 

3 

30 

27.20.2 

411 

TN 

3 

30 

27.20.3 

416 

UO 

3 

30 

27.20.6 

417 

TN 

3 

30 

27.20.5 

418 

UP 

3 

31 

27.20.7 

427 

UO 

3 

34 

27.23.2 

460* 

TN 

3 

38 

27.27.2 

494 

UO 

3 

42 

27.29.3 
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Goal  ID  and 

Completion 

OSD 

OSD 

CMC 

OSD 

CMC 

Notes 

Goal 

Time  (sec) 

ID 

Operator 

Scenario 

Page 

IPME 

Descriptor 

Segment 

Number 

Model 

Task  ID 

530* 

UO 

3 

46 

27.32.2 

531* 

TN 

3 

46 

27.32.3 

532* 

UP 

3 

47 

27.32.4 

545 

UO 

3 

46 

27.32.10 

546 

TN 

3 

46 

27.32.9 

547 

UP 

3 

47 

27.32.11 

VO 

6.9.1  VTUAV 
piloting 
aspects  are 
studied 

20 

204 

UP 

1 

7 

4.1.14.11 

7.1.3  VTUAV 

4- 

162 

UP 

1 

7 

4.1.14.12 

flight  path  is 

continuous 

220 

UP 

1 

11 

4.2.3.16 

monitored 

354 

UP 

1 

15 

4.2.6.37 

286 

UP 

1 

19 

4.2.16.27 

372 

UP 

1 

19 

4.2.16.26 

378 

UP 

1 

27 

4.2.15.2 

436 

UP 

1 

31 

4.2.26.13 

454 

UP 

1 

35 

4.2.27.1 

474 

UP 

1 

35 

4.2.18.2 

202 

TN 

2 

6 

21.17.16 

232 

UP 

2 

7 

21.17.9 

242 

UP 

2 

11 

21.17.26 

316 

UP 

2 

15 

21.20.15 

492 

TN 

2 

26 

21.26.16 

652 

UP 

2 

39 

21.27.2 

654 

UP 

2 

43 

21.27.11 

660 

TN 

2 

42 

21.36.1 

132 

UP 

3 

3 

27.4.2 

138 

UP 

3 

3 

27.4.7 

272 

UP 

3 

19 

27.38.9 

527 

UP 

3 

43 

27.31.2 

7. 1.5.6 

25 

434 

UP 

2 

19 

21.23.17 

Also  a 

VTUAV  EO 

PO  goal 

images  are 
monitored 

7. 1.9.2  initial 

12-30 

146 

UP 

1 

7 

4.1.14.15 

system  checks 

270 

UO 

3 

8 

27.38.7 

on  VTUAV 
are  conducted 

271 

UP 

3 

19 

27.38.8 
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Goal  ID  and 

Completion 

OSD 

OSD 

CMC 

OSD 

CMC 

Notes 

Goal 

Time  (sec) 

ID 

Operator 

Scenario 

Page 

IPME 

Descriptor 

Segment 

Number 

Model 

Task  ID 

7. 1.9.4 

6-cont 

256 

UP 

2 

11 

21.17.31 

VTUAV 

372 

UP 

2 

15 

21.22.8 

systems  are 
monitored 

135 

UP 

3 

3 

27.4.5 

7.3. 1.8  Mini 

10-18 

505 

UP 

2 

35 

21.31.7 

UAV  route  is 
plotted 

555 

UP 

3 

51 

27.34.5 

1 3.1 .2  initial 
system  checks 
on  Mini  UAV 
are  conducted 

4 

358 

UP 

2 

27 

21.21.16 

13.13  Mini 

9-cont 

452 

uo 

2 

30 

21.29.16 

UAV  systems 
are  monitored 

460 

UP 

2 

31 

21.29.7 

7.3. 7.4  Mini 
UAV  systems 
are  managed 

5 

458 

UP 

2 

35 

21.29.11 

9.4.1. 1 

VTUAV 
refuelling 
location  is 
transmitted 

5 

248 

UP 

1 

11 

4.1.21.8 

9.4.1.2 

VTUAV 
calculated  time 
on  task  is 
transmitted 

5 

260 

UP 

1 

11 

4.1.21.15 

9.4. 1.4  specific 

3-20 

490 

UP 

1 

35 

4.2.29.6 

information 

128 

UP 

2 

3 

21.12.3 

regarding  a 

216 

UP 

2 

7 

21.17.8 

UAV 

218 

uo 

2 

6 

21.17.14 

236 

UP 

2 

11 

21.17.10 

238 

uo 

2 

10 

21.17.15 

244 

uo 

2 

10 

21.17.24 

250 

UP 

2 

11 

21.17.28 

252 

uo 

2 

10 

21.17.30 

226 

TN 

2 

10 

21.18.6 

228 

uo 

2 

10 

21.18.7 

416 

UP 

2 

31 

21.28.2 

476 

UP 

2 

19 

21.26.2 

477 

TN 

2 

18 

21.26.10 

487 

UP 

2 

23 

21.26.7 

490 

uo 

2 

22 

21.26.9 
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Goal  ID  and 

Completion 

OSD 

OSD 

CMC 

OSD 

CMC 

Notes 

Goal 

Time  (sec) 

ID 

Operator 

Scenario 

Page 

IPME 

Descriptor 

Segment 

Number 

Model 

Task  ID 

644 

UP 

2 

39 

21.27.13 

214 

uo 

3 

6 

27.6.14 

215 

TN 

3 

6 

27.6.15 

216 

UP 

3 

7 

27.6.16 

240 

TN 

3 

14 

27.11.13 

273 

UP 

3 

19 

27.38.10 

290 

UP 

3 

15 

27.14.4 

298 

UP 

3 

19 

27.14.17 

350 

uo 

3 

26 

27.17.17 

395 

uo 

3 

26 

27.18.8 

396 

TN 

3 

26 

27.18.7 

397 

UP 

3 

27 

27.18.9 

389 

uo 

3 

30 

27.22.17 

432 

TN 

3 

34 

27.23.6 

433 

UP 

3 

35 

27.23.7 

434 

UO 

3 

34 

27.23.8 

580 

JJP 

3 

55 

27.36.8 

2.3.2  location 

X 

Not  in 

of  friendly 
units 

OSD 

2.3.4  list 

X 

Not  in 

mission 

OSD 

activities  of 
friendly  units 

2.4.2  location 

X 

Not  in 

of  neutral  units 

OSD 

2.5.2. 1 

6-16 

168 

TN 

2 

2 

21.12.13 

location  of  all 

234 

TN 

2 

6 

21.17.20 

unknown  unit 

222 

TN 

2 

10 

21.18.14 

icons  on 

584 

TN 

2 

46 

21.35.9 

tactical  plot 

2. 5. 2.2  relative 

X 

Not  in 

location  of 
own  position 
to  unknown 
units 

OSD 

2.6.2. 1 

X 

Not  in 

location  of  all 
known  terrorist 
unit  icons  on 
tactical  plot 

OSD 
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Goal  ID  and 

Completion 

OSD 

OSD 

CMC 

OSD 

CMC 

Notes 

Goal 

Time  (sec) 

ID 

Operator 

Scenario 

Page 

IPME 

Descriptor 

Segment 

Number 

Model 

Task  ID 

2. 6. 2.2  relative 

X 

Not  in 

location  of 

OSD 

own  position 

to  known 

terrorist  units 

6.4.1  the 

5-30 

456 

TN 

1 

34 

4.2.30.8 

probable 

458 

UO 

1 

34 

4.2.30.6 

position  of  the 

460 

UP 

1 

35 

4.2.30.4 

terrorist  vessel 

464 

TN 

1 

34 

4.2.30.15 

466 

TN 

1 

34 

4.2.30.14 

468 

TN 

1 

34 

4.2.30.13 

6.4.4  search 

15 

662 

TN 

2 

42 

21.36.2 

area  is 

appropriate  for 

the  current 

situation 

6.4.5  display 

12 

664 

TN 

2 

42 

21.36.3 

area  on 

workstation  is 

appropriate 

2.6.1 

X 

Not  in 

workstation  is 

OSD 

formatted  to 

display  the 

tactical  plot 

4.1.5  latest 

8-15 

282 

UO 

1 

14 

4.2.6. 1 

position  of  all 

390 

UO 

1 

30 

4.2.28.8 

unknown 

contacts  is 

plotted 

4.1.10  search 

25 

570 

TN 

3 

50 

27.35.4 

Not  in  trc 

plan  for 

airborne  threat 

is  produced 

4.2.5  contact  is 

10-cont 

375 

UO 

1 

26 

4.2.17.5 

sought  using 

380 

UO 

1 

26 

4.2.17.6 

UAV  radar 

4.2.6  contact  is 

X 

Not  in 

visually  sought 

OSD 

using  UAV  EO 

suite 
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ID 
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CMC 
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Segment 

OSD 

Page 

Number 

CMC 

IPME 

Model 

Task  ID 

Notes 

7. 1.2.1 

5-cont 

218 

UP 

1 

11 

4.2. 3. 5 

VTUAV 

352 

UP 

1 

15 

4.2.6.38 

heading  has 

284 

UP 

1 

19 

4.2.16.28 

changed  to  a 

296 

uo 

1 

14 

4.2.16.3 

new  heading 

370 

UP 

1 

19 

4.2.16.25 

376 

UP 

1 

27 

4.2.15.1 

384 

UP 

1 

27 

4.2.15.6 

400 

UP 

1 

31 

4.2.26.23 

434 

UP 

1 

31 

4.2.26.14 

452 

UP 

1 

31 

4.2.27.2 

470 

UP 

1 

35 

4.2.18.3 

196 

UP 

2 

7 

21.17.5 

264 

UP 

2 

11 

21.17.33 

292 

UP 

2 

15 

21.20.2 

314 

UP 

2 

15 

21.20.10 

485 

UP 

2 

23 

21.26.14 

640 

UP 

2 

39 

21.27.1 

137 

UP 

3 

3 

27.4.6 

524 

UP 

3 

43 

27.31.1 

7. 1.2.2 

5-6 

240 

UP 

2 

11 

21.17.11 

VTUAV 

248 

UP 

2 

11 

21.17.27 

altitude  has 

changed  to  a 

new  altitude 

7. 1.2.4 

4 

579 

UP 

3 

55 

27.36.7 

VTUAV 

autopilot  set  to 

autonomous 

mode 

7. 1.2.5 

5 

554 

UP 

3 

51 

27.33.19 

VTUAV 

transitions  to  a 

hover 

7. 1.4.1 

6-30 

148 

uo 

1 

6 

4.1.14.8 

VTUAV  EO 

178 

uo 

2 

6 

21.15.5 

sensor  settings 

486 

uo 

2 

22 

21.26.12 

are  optimized 

140 

uo 

3 

2 

27.4.12 

7. 1.5.4 

7 

428 

uo 

2 

18 

21.23.1 

Not  in  trc 

VTUAV  EO 

zoomed  in  on  a 

portion  of  boat 
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Operator 
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Page 
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Descriptor 


Segment  Number 
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7. 1.5.5 

12-cont 

430 

ruo 

2 

18 

21.23.3 

VTUAV  EO 

431 

uo 

2 

18 

21.23.4 

used  to  record 
images  of 
contact 

145 

uo 

3 

2 

27.4.11 

7. 1.5.7 

VTUAV  EO 
image  file  is 
stowed 

10 

480 

uo 

2 

22 

21.26.11 

7.1.6  VTUAV 

X 

Not  in 

radar  is 
configured 

OSD 

7. 1.7.1 

14-60 

222 

uo 

1 

10 

4.2.3.18 

VTUAV  radar 
is  used  to 
search  for  lost 

contact 

126 

uo 

2 

2 

21.12.8 

7. 1.7.2 

X 

Not  in 

VTUAV  radar 
is  used  to 
vector  another 

asset 

OSD 

7. 1.7.3 

4-10 

351 

uo 

1 

14 

4.2.6.40 

VTUAV  radar 

374 

uo 

1 

26 

4.2.17.4 

is  used  to 

200 

uo 

2 

6 

21.17.4 

GENTRACK 

contact 

258 

uo 

2 

10 

21.17.35 

7. 1.9.1 

12-cont 

144 

uo 

1 

6 

4.1.14.7 

VTUAV  data 

145 

uo 

1 

6 

4.1.14.6 

up-link  is 
maintained 

122 

TN 

2 

2 

27.2.2 

7.3. 1.3  Mini 

3-10 

224 

TN 

2 

10 

21.18.4 

UAV  waypoint 

288 

TN 

2 

22 

21.19.13 

is  inserted 

290 

TN 

2 

22 

21.19.14 

462 

UO 

2 

34 

21.29.12 

294 

TN 

3 

14 

27.14.11 

303 

TN 

3 

18 

27.14.19 

360 

TN 

3 

26 

27.19.8 

449 

? 

? 

? 

? 

479 

TN 

3 

38 

27.28.12 

7.3.1.12  Mini 
UAV  opening 
height  is  set 

7 

343 

TN 

1 

22 

4.3.9.17 
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Page 
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Descriptor 
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Task  ID 

7.3.2. 1  Mini 

6-8 

356 

UP 

2 

31 

21.21.15 

UAV  heading 

422 

UP 

2 

31 

21.28.8 

has  changed  to 

335 

UP 

3 

23 

27.17.1 

a  new  heading 

337 

UP 

3 

23 

27.17.11 

339 

UP 

3 

27 

27.17.15 

385 

UP 

3 

31 

27.22.3 

510 

UP 

3 

43 

27.30.3 

13.22  Mini 
UAV  altitude 
has  changed  to 
a  new  altitude 

22 

425 

UP 

2 

31 

21.28.21 

7. 3. 2.4  Mini 

6-8 

424 

UP 

2 

31 

21.28.22 

UAV  altitude 
change  has 
been  initiated 

553 

UP 

3 

51 

27.33.18 

132.5  Mini 

6-10 

466 

uo 

2 

34 

21.29.14 

UAV 
automatic 
over-flight 
function  is 
initiated 

506 

UP 

2 

39 

21.31.19 

13.2.6  Mini 

7-45 

544 

uo 

2 

42 

21.34.10 

UAV  is  set  to 

autonomous 

operations 

572 

uo 

3 

54 

27.36.6 

13.2.1  Mini 

5-8 

336 

UP 

3 

23 

21.17.2 

UAV  initiates 

338 

UP 

3 

27 

27.17.13 

a  pre-planned 

340 

UP 

3 

27 

27.17.4 

route  about  a 

386 

UP 

3 

31 

27.22.4 

contact 

512 

UP 

3 

43 

27.30.4 

518 

UP 

3 

47 

27.30.8 

7.3.2. 8  Mini 
UAV  initiates 
a  self-destruct 

6 

224 

UP 

3 

11 

27.8.8 

Not  in  trc 

manoeuvre 

13.2.9  manual 
control  of  Mini 
UAV  is 
initiated 

4 

399 

UP 

3 

27 

27.18.11 

7.3.2.10  Mini 

14-cont 

400 

UP 

3 

31 

27.18.12 

UAV  is 

424 

UP 

3 

35 

27.20.12 

manoeuvring 
about  contact 
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Page 
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CMC 

IPME 
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7.3.3. 1  Mini 

2-23 

346 

TN 

2 

26 

21.21.11 

UAV  symbol 

348 

UP 

2 

27 

21.21.10 

has  appeared 

319 

TN 

3 

22 

27.15.7 

on  the  surface 

320 

UO 

3 

22 

27.15.6* 

plot 

321 

UP 

3 

22 

27.15.8 

362 

TN 

3 

26 

27.19.7 

363 

UO 

3 

26 

27.19.6 

364 

UP 

3 

27 

27.19.9 

482 

TN 

3 

38 

27.28.13 

483 

UO 

3 

38 

27.28.11 

484 

UP 

3 

39 

27.28.16 

7.3. 3.2  Mini 

5 

350 

UP 

2 

27 

21.21.12 

UAV  is  in 

descent 

following 

deployment 

1.3.33  Mini 

Cont 

508 

UP 

2 

39 

21.31.20 

UAV  is 

342 

? 

? 

? 

? 

following  the 

387 

UP 

3 

31 

27.22.9 

planned  flight 

514 

UP 

3 

43 

27.30.5 

path 

519 

UP 

3 

47 

27.30.9 

7.3. 3.4  Mini 

5-6 

352 

UP 

2 

27 

21.21.13 

UAV  is 

426 

UP 

2 

31 

21.28.20 

establishing 

level  flight 

7. 3. 3. 5  Mini 

13 

592 

TN 

2 

46 

21.35.15 

UAV  is 

autonomously 

following 

contact 

7.3.4. 1  Mini 

10 

345 

UO 

3 

22 

27.17.5 

UAV  EO 

346 

UO 

3 

22 

27.17.10 

sensor  settings 

347 

UO 

3 

26 

27.17.14 

are  optimized 

388 

UO 

3 

30 

27.22.2 

516 

UO 

3 

42 

27.30.2 

82 


DRDC  CR  2009-059 


Goal  ID  and 
Goal 

Descriptor 

Completion 
Time  (sec) 

OSD 

ID 

OSD 

Operator 

CMC 

Scenario 

Segment 

OSD 

Page 

Number 

CMC 

IPME 

Model 

Task  ID 

Notes 

7.3. 5.2  Mini 

4-cont 

521 

uo 

2 

42 

21.31.21 

UAVEO 

522 

UP 

2 

39 

21.31.18 

sensor  is  used 

586 

TN 

2 

46 

21.35.10 

to  study  a 

591 

TN 

2 

46 

21.35.14 

contact 

380 

UO 

3 

26 

27.18.4 

381 

TN 

3 

26 

27.18.5 

382 

UP 

3 

27 

27.18.10 

402 

UO 

3 

30 

27.18.15 

403 

TN 

3 

30 

27.18.14 

404 

UP 

3 

31 

27.18.13 

405 

UO 

3 

30 

27.18.18 

406 

TN 

3 

30 

27.18.17 

407 

UP 

3 

31 

27.18.16 

425 

UO 

3 

30 

27.20.13 

560 

UO 

3 

46 

27.34.3 

564 

UO 

3 

50 

27.34.9 

565 

TN 

3 

50 

27.34.8 

566 

UP 

3 

51 

27.34.7 

584 

UO 

3 

54 

27.36.13 

7.3. 5.4  Mini 

12 

518 

UO 

2 

38 

21.31.15 

UAV  EO 

zoomed  in  on  a 

portion  of  boat 

7.3. 5.6  Mini 

14-20 

375 

UO 

3 

26 

27.18.1 

UAVEO 

376 

TN 

3 

26 

27.18.2 

sensor  is  used 

377 

UP 

3 

27 

27.18.3 

to  track  a 

520 

UO 

3 

46 

27.30.7 

contact 

13.5.1  Mini 

8-15 

360 

UO 

2 

14 

21.22.1 

UAV  EO  is 

510 

UO 

2 

34 

21.31.8 

used  to  record 

519 

UO 

2 

38 

21.31.16 

high  definition 

520 

UO 

2 

38 

21.31.17 

images 

7.3. 5. 8  Mini 

5 

464 

UO 

2 

34 

21.29.13 

UAV  FOV 

trapezoid  is 

over  contact 

7.3. 5.9  Mini 

9 

392 

UO 

3 

30 

27.22.8 

UAVEO 

video 

recording  is 

operating 
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ID 
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Page 
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7.3.5.10  EO 

Cont 

463 

TN 

3 

38 

27.27.4 

images  are 

organized  on 

desktop 

10.1.2.1  MC 

10 

154 

TN 

1 

2 

4.1.18.1 

workstation  is 

configured 

10.1.2.2  VO 

10 

160 

UP 

1 

7 

4.1.18.7 

workstation  is 

configured 

10.1.2.3  PO 

10 

156 

uo 

1 

6 

4.1.18.12 

workstation  is 

configured 

4.4 

X 

Not  in 

identification 

OSD 

of  contacts 

(auto) 

4.4.5  crew 

5-cont 

358 

uo 

1 

18 

4.2.16.8 

identifying 

389 

uo 

1 

26 

4.2.28.9 

vessel  using 

440 

uo 

1 

30 

4.2.26.10 

UAV  EO  suite 

500 

uo 

2 

34 

21.31.38 

523 

uo 

2 

42 

21.31.22 

528 

uo 

2 

42 

21.31.26 

4.4.6  ISAR 

6 

610 

? 

? 

? 

? 

imagery  is 

151 

TN 

3 

6 

27.5.11 

downloaded 

and  analysed 

4.4.7  UAV 

8-14 

362 

UO 

2 

14 

21.22.2 

images  of  boat 

511 

uo 

2 

34 

21.31.10 

are  compared 

with  database 

4.4.8  crew 

5-12 

364 

uo 

2 

14 

21.22.3 

classify  vessel 

512 

uo 

2 

34 

21.31.11 

using  UAV  EO 

suite 

4.4.9  legality 

6-14 

535 

uo 

2 

42 

21.34.1 

of  fishing  boat 

589 

TN 

2 

46 

21.35.12 

activities 

4.4.10  contact 

4 

590 

TN 

2 

46 

21.35.13 

can  be 

identified 
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7.1.1  VTUAV 

X 

Not  in 

navigation  is 
conducted 

OSD 

(manual) 

7.1. 1.1 

5-30 

140 

TN 

1 

6 

4.2.4. 1 

VTUAV  route 

120 

TN 

2 

2 

21.19.9 

to  the  next 

176 

UP 

2 

7 

21.15.9 

operating  area 

212 

UP 

2 

7 

21.17.6 

is  planned 

134 

UP 

3 

3 

27.4.4 

525 

UP 

3 

43 

27.31.3 

578 

UP 

3 

55 

27.36.5 

7.1. 1.2 

X 

Not  in 

VTUAV 
search  pattern 
is  planned 

OSD 

7.1. 1.2.1 

7-15 

382 

UP 

1 

27 

4.2.15.5 

selection  of  an 
appropriate 
search  pattern 

482 

UP 

2 

23 

21.26.4 

7.1. 1.2.7 
location  of 
contact  symbol 
is  determined 
on  tacplot 

8 

475 

UP 

2 

19 

21.26.1 

7.1. 1.2.8 

3 

481 

X 

Not  in 

direction  of 
movement  of 
contact  symbol 

OSD 

7.1. 1.3 

3-15 

208 

TN 

1 

6 

4.2. 4.2 

VTUAV 

404 

TN 

1 

30 

4.2.26.25 

waypoint  is 

194 

UP 

2 

7 

21.17.2 

inserted 

484 

UP 

2 

23 

21.26.6 

481 

TN 

3 

38 

27.28.17 

7.1. 1.6 

X 

Not  in 

VTUAV  time 
on  task  is 
calculated 

OSD 

7.1. 1.6.1 

location  of 

potential 

VTUAV 

refuelling 

platforms 

20 

247 

UP 

1 

7 

4.1.21.6 
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Goal  Time  (sec)  ID  Operator 

Descriptor 

CMC  OSD 

Scenario  Page 
Segment  Number 

CMC  Notes 

IPME 

Model 

Task  ID 

7.1. 1.6.2 
estimated  CPF 
location  at 
time  off  task 

8 

254 

UP 

1 

11 

4.1.21.9 

7.1. 1.6.3 
VTUAV  fuel 
on  board  and 
average  fuel 
flow 

5 

256 

UP 

1 

7 

4.1.21.7 

7. 1.1. 6.4  rough 
VTUAV  time 
on  task  is 
calculated 

4 

246 

UP 

1 

7 

4.1.21.2 

7.1. 1.6.5 
precise 

VTUAV  time 
on  task  is 
calculated 

10-20 

258 

486 

UP 

UP 

1 

1 

11 

35 

4.1.21.14 

4.2.29.8 

7. 1.1. 6. 6  any 
problems 
associated  with 
refuelling 

10 

488 

UP 

1 

35 

4.2.29.7 

7.1. 1.8  10  210  TN  1  10  4.2. 4. 3 

VTUAV  221  UO  1  10  4.2.4.37 

activities  are 

planned 

7.1. 1.9  10  223  UP  1  11  4.2.4.35 

VTUAV 
planning 
activities  are 
monitored 

7.1.1.10  12  214  UP  2  7  21.17.7 

VTUAV  route  483  UP  2  23  21.26.5 

is  plotted 

7.1.1.11  7  590  TN  3  54  27.36.11 

handover  of 

VTUAV  has 
been  prepared 
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List  of  symbols/abbreviations/acronyms/initialisms 


AF 

Air  Force 

ASO 

Acoustic  Sensor  Operator 

CAMA 

CF  AF  MOSID  Application 

CF 

Canadian  Forces 

CFEC 

Canadian  Forces  Experimentation  Centre 

CPF 

Canadian  Patrol  Frigate 

CHOGM 

Commonwealth  Heads  of  Government  Meeting 

DND 

Department  of  National  Defence 

DRDC 

Defence  Research  &  Development  Canada 

DRDKIM 

Director  Research  and  Development  Knowledge  and  Information 

Management 

HR 

Human  Resources 

HR(mil) 

Human  Resources  -  Military 

HSI 

Human  Systems  Integration 

IPME 

Integrated  Performance  Modelling  Environment 

ISAR 

Inverse  Synthetic  Aperture  Radar 

ISMAT 

Integrated  Simulation  Manpower  Analysis  Tool 

JATO 

Jet  Assist  Take-Off 

JSI 

Job  Similarity  Index 

LCol 

Lieutenant  Colonel 

MALE 

Medium  Altitude  Long  Endurance 

MOC 

Military  Occupational  Code 

MANPRINT 

MANpower  and  PeRsonnel  INTegration 

MOS 

Military  Occupational  Structure 

MOSART 

Military  Occupational  Structure  Analysis,  Redesign,  and  Tailoring 

MOSID 

Military  Occupational  Structure  Identification 

NASO 

Non-Accoustic  Sensor  Operator 

OSD 

Operational  Sequence  Diagram 

PSF 

Performance  Shaping  Function 
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R&D 

Research  &  Development 

ROC 

Regional  Operations  Centre 

SA 

Scientific  Authority 

SME 

Subject  Matter  Expert 

TACNAV 

Tacital  Navigator 

TK 

Task  and  Knowledge  statement 

TSK 

Task,  Skill,  and  Knowledge  statement 

UAV 

Uninhabited  Air  Vehicle 

US 

United  States 
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